Message Boards Message Boards

LibraryLink: The purpose of "Shared" passing

GROUPS:

Hello everyone,

I am looking for a (hopefully official) answer about the intended purpose of "Shared" passing in LibraryLink. LibraryLink developers have given me very useful responses here on Wolfram Community in the past.

Basically, I would like to know if it Shared passing is suitable for returning multiple results using a single library function call.

In plain C, if I wanted to return multiple results from a single function, I would use pass-by-reference (i.e. pointers) and modify multiple function arguments. We can do the same thing using "Shared" passing in LibraryLink, and it appears to work, but there are many caveats (I described some here). Therefore I am not sure if it is a good idea to do this.

I do understand that one purpose of "Shared" is to avoid copying the data, but "Constant" also does that. Another purpose is to maintain ownership of the tensor on the library side. I am not sure when this would be needed if the tensor was not originally created by the library. I wonder if another purpose is to allow modifying arguments?

It seems to me that none of the MTensor examples that come with LibraryLink show such a usage. However, there is at least one MSparseArray example which shows in-place modification of a sparse array.


Finally, I have a specific concern about this type of use of "Shared". One problem with such in-place modification of tensors in LibraryLink is that two values which are conceptually separate in Mathematica may share storage under the hood. The two values may reference the same piece of memory. If one value is modified in Mathematica, it is automatically copied. If it is modified in C / LibraryLink, it is not copied. Thus we must be very careful not to modify a tensor in LibraryLink if multiple Mathematica-side values are referencing it.

My concern is that I am not certain that we can prevent such a situation with certainty. I can create a packed array manually, and try to avoid aliasing it. But Mathematica may decide to do its own optimizations which are out of my control. For example, the Share[] command will find identical values and share their storage. Does it do this with packed arrays too? Does Share[] ever run automatically? If yes, then I would not be able to prevent aliasing, and my program may break in very subtle ways.

These sorts of concerns are why I have not yet started using Shared passing this way. I hope my question above is clear. If not, let me know.

POSTED BY: Szabolcs Horvát
Answer
6 months ago

Group Abstract Group Abstract