# Changes in MathLink in Version 10 ? **URL:** \ **Category:** Questions\ **Tags:** software-development, mathematica, external-interfaces\ **Created:** [July 15, 2014, 1:07pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678 "2014-07-15T13:07:27Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![fkampas](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/fkampas/32/41217_2.png) [@fkampas](https://community.wolfram.com/u/fkampas)\ **Post date:** [July 15, 2014, 1:07pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/1 "2014-07-15T13:07:27Z")
In Version 10, MathLink is now called the “Wolfram Symbolic Transfer Protocol”. Is there new functionality or is it just a name change? ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 15, 2014, 2:30pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/2 "2014-07-15T14:30:30Z")
The interface level has been bumped to 4. If you look in `mathlink.h`, you’ll find a list of new functions and a list of removed functions (quoted at the end). The functions can be looked up in the documentation. There’s probably more to say here, and I’m curious about the changes too. I also noticed that parallel kernels launch _much_ faster in version 10, which is likely due to MathLink changes. * * * The name chage, in my opinion, is pretty counterproductive. Renaming all API functions (used for engineering) just because of a marketing decision (!!) to rebrand the system is just wrong and I don’t see how it could bring aything but trouble … Now Mathematica comes with two separate directories, one for MathLink and one for WSTP. The two look identical other than the renaming. One contains the libraries and header files with the ML prefixed functions, the other one has the WS prefixed functions. * * * ``` /* * Interface 4 adds the following exported functions: * * MLCreateLinkWithExternalProtocol * MLDoNotHandleSignalParameter * MLEnableLinkLock * MLFilterArgv * MLGetAvailableLinkProtocolNames * MLGetInteger8 * MLGetInteger8Array * MLGetInteger8ArrayData * MLGetInteger8List * MLGetLinksFromEnvironment * MLGetNumberAsByteString * MLGetNumberAsString * MLGetNumberAsUCS2String * MLGetNumberAsUTF16String * MLGetNumberAsUTF32String * MLGetNumberAsUTF8String * MLHandleSignal * MLIsLinkLoopback * MLLinkEnvironment * MLLogFileNameForLink * MLLogStreamToFile * MLLowLevelDeviceName * MLPutInteger8 * MLPutInteger8Array * MLPutInteger8ArrayData * MLPutInteger8List * MLPutRealNumberAsUCS2String * MLPutRealNumberAsUTF16String * MLPutRealNumberAsUTF32String * MLPutRealNumberAsUTF8String * MLReleaseInteger8Array * MLReleaseInteger8List * MLReleaseLinkProtocolNames * MLReleaseLinksFromEnvironment * MLReleaseLogFileNameForLink * MLReleaseLowLevelDeviceName * MLReleaseParameters * MLSetThreadSafeLinksParameter * MLStopHandlingSignal * MLStopLoggingStream * MLValid * MLWaitForLinkActivity * MLWaitForLinkActivityWithCallback * * Interface 4 removes the following API functions. * .... */ ``` ---
**Author:** ![fkampas](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/fkampas/32/41217_2.png) [@fkampas](https://community.wolfram.com/u/fkampas)\ **Post date:** [July 15, 2014, 2:34pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/3 "2014-07-15T14:34:01Z")
What does “the interface level has been bumped to 4” mean? ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 15, 2014, 2:42pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/4 "2014-07-15T14:42:04Z")
I should have probably called it interface version 4, I’m not sure which is the official term used by WRI. It just means that the version number has been changed, which means that compatiblity with other interface versions is not guaranteed (new functions may be added, old functions removed or the calling syntax may change). There is a seprate versioning for the public documented MathLink C interface and for the MathLink library. When the MathLink library gets a bugfix, the API to access it does not necessarily change. The API didn’t change between Mathematica 8 and Mathematica 9, to my knowledge. But don’t quote me on this, I’m not an expert on the topic. ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 15, 2014, 3:54pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/5 "2014-07-15T15:54:53Z")
You can find a helpful description of the changes in Interface 4 at [WSTP Interface 4—Wolfram Documentation](http://reference.wolfram.com/language/tutorial/WSTPInterface4.html). ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 15, 2014, 3:56pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/6 "2014-07-15T15:56:47Z")
This is very useful. I didn’t find this page myself. ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 15, 2014, 4:00pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/7 "2014-07-15T16:00:04Z")
Is there an option for `mprep` that causes it to output Interface 3 compatible code? Also, have you seen this type of linking error with v10 on Linux? `libML64i3.so: undefined reference to `uuid_parse'` ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 15, 2014, 4:08pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/8 "2014-07-15T16:08:17Z")
Mprep should produce code that will work with the interface 3 version of the libraries. When building the code, if you want to link with the interface 3 library, you will need to define the preprocessor macro MLINTERFACE to 3, rather than letting it default to 4 in mathlink.h. The linker error you are seeing that references uuid\_parse is coming from a new requirement int he MathLink libraries to link with libuuid on Linux. You can use your Linux distribution’s libuuid if you have one available or can install one from the distribution’s package manager, or there is a libuuid in ${InstallationDirectory}/SystemFiles/Libraries/Linux-x86-64 suitable for use with the MathLink library. ---
**Author:** ![fkampas](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/fkampas/32/41217_2.png) [@fkampas](https://community.wolfram.com/u/fkampas)\ **Post date:** [July 15, 2014, 4:12pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/9 "2014-07-15T16:12:49Z")
Some LibraryLink code I wrote using MathLink to transfer data still works in Version 10, even though the functions have been deprecated. I imagine that means it won’t work when Version 11 comes out. ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 19, 2014, 6:12pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/10 "2014-07-19T18:12:32Z")
Thank you for the response! Will `Install`able MathLink programs compiled with the default `MLINTERFACE 4` and linked against `ML64i4` still work with earlier Mathematica versions? For example, let’s assume that we have the source code for an installable MathLink program that worked with some older version of Mathematica. Now I re-run `mprep` and re-compile the program, linking it against `ML64i4` or `ML32i4`. Will the resulting binary be usable with older versions of Mathematica, e.g. Mathematcia 8 or 9? * * * I know that I can `#define MLINTERFACE 3` and link against `ML64i3` for compatibility, but the question is whether this is necessary? ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 19, 2014, 8:27pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/11 "2014-07-19T20:27:00Z")
The programs compiled and linked with the interface 4 version of the library should indeed work with older versions of Mathematica. I assume at some point you will run into issues such as older operating systems that will eventually result in incompatibilities the further you go back with your Mathematica revisions, but if you can run the two copies of Mathematica on the same machine, then your newer binary should work with the older Kernel. ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 19, 2014, 8:29pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/12 "2014-07-19T20:29:14Z")
The LibraryLink interface that includes MathLink is very much tied to the interface used internally by the Kernel. If the Kernel is using MathLink interface 4, then the LibraryLink dlls need updated to support interface 4. The deprecated functions may continue to work for a time, however, as all things deprecated go, WRI may eventually remove those API functions from the interface altogether in a future release. ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 19, 2014, 11:13pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/13 "2014-07-19T23:13:16Z")
Thanks for the reply! Then from now on I’ll distribute binaries linked against interface 4. Some more questions, just out of curiosity: - How does the new “IntraProcess” protocol perform for transferring large data? I’m asking because LibraryLink currently doesn’t allow returning complex datatypes. For example, it’s not possible to return _both_ a matrix and a tensor from the _same_ function, and returning them via multiple LibraryLink functions has the potential to introduce a mess (it forces me to maintain a library state, which isn’t a good fit for Mathematica as it prefers side-effect-less functions). I think transferring very large packed arrays was already quite fast with MathLink with the “SharedMemory” protocol, but I’m not sure about many small arrays (or the overhead for each transfer). - Did MathLink receive any performance improvements in version 10, and generally does it have any advantages to link against interface 4 if I’m not using any interface 4 only features? ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 20, 2014, 12:02am UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/14 "2014-07-20T00:02:20Z")
@SteveWilson I’m right now investigating a bug report for [MATLink](http://matlink.org/) on Linux. It appears that cycling a string through MATLink appends a zero character code at the end somehow, on Linux only (but not on OS X). I still haven’t gotten to the bottom of the issue, and it is possible that I’m using the MathLink API incorrectly, but it looks more like a bug. I’m going to report it once I have had the time to create a minimal test case, but I wanted to give you advance warning. ![Screenshot2014-07-1919.52.52.png](https://global.discourse-cdn.com/wolfram/original/3X/0/6/06c385b727583ef7fd407c72cd3de1d02d1e1bed.png) The zero is _likely_ appended when _sending_ the string through MathLink the first time, not when receiving it back the Mma kernel. The sending function is as simple as this: ``` void eng_make_String(const unsigned short *str, int len, int characters) { mwSize mbDims[2] = {1, len}; // use len, not characters, because no support for 4-byte characters in either Mma 9 or MATLAB mxArray *var = mxCreateCharArray(2, mbDims); std::copy(str, str+len, (unsigned short *) mxGetChars(var)); returnHandle(var); } ``` with template: ``` :Begin: :Function: eng_make_String :Pattern: engMakeString[string_] :Arguments: {string} :ArgumentTypes: {UTF16String} :ReturnType: Manual :End: ``` (You can actually ignore the guts of the functions, as it’s MATLAB-specific code, but I thought a bit of context is always good. Also, I’m not sure it’s still true that MathLink doesn’t support UTF16 characters that need to be stored on 4 (not 2) bytes). ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 20, 2014, 3:40pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/15 "2014-07-20T15:40:39Z")
@SteveWilson I haven’t tracked the problem down fully yet, and I don’t yet know why it only affects MATLink on Linux but not OS X. But it looks like it’s related to the fact that now transferring an UTF16String requires a byte order mark at the beginning. I should have read through the documentation page you linked to to notice this change, though maybe it would be good to have a documentation page dedicated to incompatible changes, or “how to upgrade”. Because there’s the byte order mark, now the “length” of a UTF8String is one greater than before. The problem appears when trying to use MLINTERFACE=3, as the “length” is still one greater, but now there is no byte order mark! Here’s an example, tested on OS X with v9 and v10, compiled with the v10 MathLink libraries. ``` :Begin: :Function: cycle_string :Pattern: cycleString[s_String] :Arguments: { s } :ArgumentTypes: { UTF16String } :ReturnType: Manual :End: #include "mathlink.h" void cycle_string(unsigned short *str, int len, int characters) { MLPutFunction(stdlink, "List", 4); MLPutUTF16String(stdlink, str, len); MLPutInteger16List(stdlink, str, len); // sloppily ignore signed/unsigned difference MLPutInteger32(stdlink, len); MLPutInteger32(stdlink, characters); } int main(int argc, char* argv[]) { return MLMain(argc, argv); } ``` When I compile and link with interface version 4, this is what we get, _both_ in v10 and v9: ``` In[3]:= cycleString["asd"] Out[3]= {"asd", {-257, 97, 115, 100}, 4, 3} ``` When I compile and link with interface version 3, i.e. `-DMLINTERFACE=3 -lMLi3`, but use the libraries that come with v10, then I get this: ``` In[3]:= cycleString["asd"] Out[3]= {"asd\.00", {97, 115, 100, 0}, 4, 3} ``` The `"\.00"` part is not displayed in the front end, it’s invisible. It can only be detected with `ToCharacterCode`. This result is incorrect and I believe that it is a bug. It definitely causes incompatibilities with MathLink programs written for v9. When I compile with the libraries that come with v9, I get: ``` In[3]:= cycleString["asd"] Out[3]= {"asd", {97, 115, 100}, 3, 3} ``` For some reason this broke [MATLink](http://matlink.org/) on Linux only but not on OS X, I’m not yet sure why. I am not yet sure if it is MATLink/MATLAB that’s different between Linux and OS X or there’s also a difference in the MathLink libraries that come with v10. I have not yet reported this to Wolfram Support. I would like to wait for your input first. * * * **Update:** I have to correct myself: on OS X it might affect MATLink too, but we distribute compiled binaries on OS X, and it seems I only tested with v9-compiled binaries in v10 final. On Linux MATLink is always compiled from source so the problem came out. ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 21, 2014, 5:44pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/16 "2014-07-21T17:44:51Z")
- The IntraProcess protocol is designed specifically for communication between threads in the same process. The protocol should be quite fast. - MathLink did receive performance enhancements in Version 10 with respect to string and symbol transfers. These enhancements are automatically enabled with communicating with another Version 10 MathLink library and disabled when communicating with an older version of the library. ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 21, 2014, 5:47pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/17 "2014-07-21T17:47:07Z")
Szabolcs, I am very interested in working with this issue, however a death in the family will prevent me from intensively scrutinizing the issue until next week when I return from dealing with family issues. If you can isolate the problem prior to that time, please feel free to submit the issue to Wolfram Support, otherwise I will dive into the problem as soon as I return. I don’t normally share private details on public forums, but I wanted you to know why I will not be answering this question for a period of time. Thanks, Steve ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 21, 2014, 6:03pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/18 "2014-07-21T18:03:10Z")
Steve, I am sorry for your loss. I will summarize my findings and send them to support next weekend, so you can receive them next week. ---
**Author:** ![stevew](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/stevew/32/28502_2.png) [@stevew](https://community.wolfram.com/u/stevew)\ **Post date:** [July 29, 2014, 7:13pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/19 "2014-07-29T19:13:02Z")
Szabolcs, I just wanted to confirm for you that you have indeed found a bug. I am working on applying a fix now. Thanks, Steve ---
**Author:** ![szhorvat](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/szhorvat/32/3531_2.png) [@szhorvat](https://community.wolfram.com/u/szhorvat)\ **Post date:** [July 29, 2014, 7:36pm UTC](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678/20 "2014-07-29T19:36:33Z")
Hi Steve, Could you please clarify the precise meaning of `length` and `characters` for both interface version 3 and version 4 when passing an `UTF16String`? (as defined [here](http://reference.wolfram.com/language/ref/file/file.tm.html), under Details) Specifically, under what circumstances will these two lengths differ from each other? (Assuming that the bug is fixed, I’m not asking about the wrong behaviour.) UTF16 can store a single character either on 2 bytes or 4 bytes. Is this the only reason why `characters` can differ from `length`? Does MathLink support 4 byte characters with interface version 3? What about interface version 4? And what about Mathematica itself? (I could imagine that MathLink already supports 4 byte characters, but Mathematica doesn’t yet.) I am not very familiar with UTF16 and surrogate pairs, and I want to avoid dealing with surrogate pairs as much as possible in MATLink because (I think) MATLAB doesn’t support them anyway. The reason why I am asking these questions is so that I can avoid any crashes or misbehaviour in my program when `length` and `characters` don’t match. [Next page](https://community.wolfram.com/t/changes-in-mathlink-in-version-10/2678.md?page=2)