Your area element calculation is wrong. It should be something like this:
r = {x, y, x y};
jacobian = D[r, {{x, y}}]
areaElement = Simplify[Norm@Minors[jacobian, 2],
Element[x | y, Reals]]
NIntegrate[(x + y^2 + x^3 y^3 ) areaElement, {y, 0, 1}, {x, 0, y}]