You can in fact build a 10.3-compatible version with 10.4, it's just slightly ugly. What you need to do is replace
-F"/Applications/Mathematica 10.4.app/Contents/SystemFiles/Links/MathLink/DeveloperKit/MacOSX-x86-64/CompilerAdditions" -framework "mathlink"
with
-Wl,"/Applications/Mathematica 10.4.app/Contents/SystemFiles/Links/MathLink/DeveloperKit/MacOSX-x86-64/CompilerAdditions/mathlink.framework/Versions/4.25/mathlink"
so that you're explictly linking in the libstdc++ version of MathLink rather than the libc++ version (called 4.36). This is not at all obvious from Apple's documentation, and we had to experiment internally to make sure this worked prior to shipping 10.4. I'm glad we did so that I had this answer handy for you. :)