@SteveWilson
I haven't tracked the problem down fully yet, and I don't yet know why it only affects MATLink on Linux but not OS X.
But it looks like it's related to the fact that now transferring an UTF16String requires a byte order mark at the beginning. I should have read through the documentation page you linked to to notice this change, though maybe it would be good to have a documentation page dedicated to incompatible changes, or "how to upgrade".
Because there's the byte order mark, now the "length" of a UTF8String is one greater than before. The problem appears when trying to use MLINTERFACE=3, as the "length" is still one greater, but now there is no byte order mark!
Here's an example, tested on OS X with v9 and v10, compiled with the v10 MathLink libraries.
:Begin:
:Function: cycle_string
:Pattern: cycleString[s_String]
:Arguments: { s }
:ArgumentTypes: { UTF16String }
:ReturnType: Manual
:End:
#include "mathlink.h"
void cycle_string(unsigned short *str, int len, int characters) {
MLPutFunction(stdlink, "List", 4);
MLPutUTF16String(stdlink, str, len);
MLPutInteger16List(stdlink, str, len); // sloppily ignore signed/unsigned difference
MLPutInteger32(stdlink, len);
MLPutInteger32(stdlink, characters);
}
int main(int argc, char* argv[]) {
return MLMain(argc, argv);
}
When I compile and link with interface version 4, this is what we get, both in v10 and v9:
In[3]:= cycleString["asd"]
Out[3]= {"asd", {-257, 97, 115, 100}, 4, 3}
When I compile and link with interface version 3, i.e. -DMLINTERFACE=3 -lMLi3, but use the libraries that come with v10, then I get this:
In[3]:= cycleString["asd"]
Out[3]= {"asd\.00", {97, 115, 100, 0}, 4, 3}
The "\.00" part is not displayed in the front end, it's invisible. It can only be detected with ToCharacterCode. This result is incorrect and I believe that it is a bug. It definitely causes incompatibilities with MathLink programs written for v9.
When I compile with the libraries that come with v9, I get:
In[3]:= cycleString["asd"]
Out[3]= {"asd", {97, 115, 100}, 3, 3}
For some reason this broke MATLink on Linux only but not on OS X, I'm not yet sure why. I am not yet sure if it is MATLink/MATLAB that's different between Linux and OS X or there's also a difference in the MathLink libraries that come with v10.
I have not yet reported this to Wolfram Support. I would like to wait for your input first.
Update: I have to correct myself: on OS X it might affect MATLink too, but we distribute compiled binaries on OS X, and it seems I only tested with v9-compiled binaries in v10 final. On Linux MATLink is always compiled from source so the problem came out.
