@Szabolcs Horvat This is an issue that affects Mac OS X only and should not occur again for the foreseeable future. As @Ilian Gachevski points out, this has to do with Mathematica switching C++ standard libraries on Mac. This is a one-time change we had to make following Apple dropping support for the GNU libstdc++, which was affecting our ability to use modern C++ features and link to modern interfaces and features targeting Mac OS X 10.9 and above.

Now, it might not seem like an issue if you're writing your LibraryLink code in straight C, but anything that the CCompilerDriver builds is going to be linked against MathLink, which is a C++ library. When you're loading LibraryLink code, your library is part of Mathematica's process and needs to basically conform to the same toolchain that Mathematica used-- basic C++ types can appear corrupt when passed between these separate C++ libraries. We use the CCompilerDriver to help users follow our platform as closely as possible (which is why they're producing a different binary in these two versions).

For this particular issue, I would recommend building one instance of your library with 10.3.x and another instance with 10.4 and loading the 10.3 instance if $VersionNumber is less than 10.4 to choose which file to load with LibraryLoad. Once more, this should not affect Linux or Windows. It also will not affect code outside the process (code built with MathLink and loaded with Install or built separately and loaded with ProcessLink).

Please let me know if you have any additional questions about this problem or if there's any other guidance I can provide on how to work around this.

If I only have M10.4 installed, can I compile a binary that will work with M10.3? To make the question more concrete, suppose I use the following command line (slightly modified from what CreateLibrary uses):

I don't think so. We provide runtime compatibility with the older libstdc++-based MathLink framework, but OS X doesn't provide a great way to select versioned interfaces at link-time. For the Wolfram RTL, we don't provide a libstdc++-based library with this build. If the versioning issue ever becomes prohibitive, I think we could see about providing tech support with a package containing only the link-time dependencies from 10.3, then it would be as simple as switching C compiler flags.

Really, I recommend keeping 10.3 for as long as you plan to support it with your LibraryLink code. That's the simplest solution. (or at least keeping its mathlink framework, SystemFiles/Includes directory, and WolframRTL libraries from SystemFiles/Libraries-- that's all you need!).

With Mathematica 10.3, I compiled some libraries (where I used C++11) using -mmacosx-version-min=10.9. Without this I couldn't use C++11 features ... The result did work so far. But could this, in principle, cause problem with 10.3 (which uses -mmacosx-version-min=10.6 by default)? Does -mmacosx-version-min=10.9 imply anything about libc++ vs libstdc++?

The Mac OS X compatibility flag means exactly what it sounds like it means. If you're targeting only 10.9 and above, it shouldn't be dangerous for you to use whatever flag in principle. It does imply -stdlib=libc++, so make certain that the library your produced didn't link against libc++:

$ otool -L /path/to/libmylibrary.dylib

There are some C++11 features in libstdc++, but it's version 4.2.x and it's definitely not feature complete (the compiler should complain if you try to use something that isn't present-- I think -std=C++11 or -std=C++0x may also be necessary in that case). Both libstdc++ and libc++ have backward compatibility, so linking against different versions of those libraries and headers should be safe (using the Mac OS X compatibility flags) provided you don't use the wrong one.

Really, Mac OS X 10.9 with libc++ is where Apple got their act together on C++11, so I highly recommend targeting that platform and beyond with your C++11 code.

I believe that if you add the following extension to your PacletInfo.m, the $SystemID value is significant:

Paclet[
        Name -> "My Paclet",
        Version -> "x.x.x",
        Extensions -> {
            {"LibraryLink", Root->"LibraryResources"}
        }
]

This will tell FindLibrary where to find your libraries (informed by the $SystemID). Using a full path is perfectly valid for corner cases like this, though.

Thank you, I'm going to try this! :)

I'm curious, why is the 4.25 (libstdc++) version included with 10.4? Is it for compatibility with certain MathLink programs which run as a separate process (but perhaps not for use with LibraryLink which runs in the kernel process)? Given that it is included, is it still expected that the LibraryLink binary I compile with 10.3 does load in 10.4, but then crashes sporadically (so far I only see a crash when calling MLNextPacket())?

This is more out of curiosity (and wanting to make sure that nothing is broken in the 4.25 version), as I already modified my package to have two binaries on OS X. I'm not looking to use a single one.

@Szabolcs I am afraid maintaining two separate versions of the library on OS X may be unavoidable, this is caused by the libc++ vs libstdc++ ABI incompatibility.

The problem exists only on OS X

Yes, correct.

There's only one break in compatibility, i.e. all 10.x for x?3 and again all 10.x for x?4 are going to be compatible with each other too?

That should largely be true for the 10.x series of products, but I don't think we guarantee perfect binary compatibility between versions with MathLink or the Wolfram RTL.

(you seem to have replied elsewhere, so I'll reply there)

Hi Szabolcs,

How about your MATLink package, does that use LibraryLink under the hood? I'm using it now, and will go to 10.4 soon as well... or should I perhaps keep both versions installed?

--SH

Reported one of the two problems (libraries compiled with 10.4 don't load in 10.3) as CASE:3544676.

The test case I sent: https://dl.dropboxusercontent.com/u/38623/Forward-compat.zip

Still working on a test case for the other problem (which is much more involved).