Message Boards Message Boards

Import a dll function in Mathematica?

GROUPS:

I read that it's possible import dll function in Mathematica. Now I ask you a question: suppose I have a c function

float * function(float * vertex, int * face)
{ 
       ....
       int size=getSize(vertex);
       float * r=new float[size];
       return r;    
}

Can I allocate memory inside dll and return data in Mathematica? How can I call a similar function from Mathematica? Is it possible to pass the result as an argument? Thanks in advance.

POSTED BY: Alessio Alessio
Answer
12 days ago

If you write the C code yourself, the best way is using LibraryLink. Please see my old question about this:

In particular, LibraryLink is the only method that allows you to allocate memory yourself (indirectly), then pass this data to Mathematica without copying and let Mathematica manage the memory from there on.

If you use the method linked by Frank, you will still be responsible for freeing that memory. (To be honest, I am not certain how this would work exactly, as it requires .NET expertise).

POSTED BY: Szabolcs Horvát
Answer
12 days ago

Sorry where can I find some complete example with c code and mathematica code. I forget to say that my dll body function need to call some template c++ code. Now I ask you if NETLink or LibraryLink is again a solution.

float * function(float * vertex, int * face)
{ 
       // call template c++ code
       int size=getSize(vertex);
       float * r=new float[size];
       return r;    
}
POSTED BY: Alessio Alessio
Answer
11 days ago

Sorry where can I find some complete example with c code and mathematica code.

I just gave you a link with several complete examples ... did you look at it?

POSTED BY: Szabolcs Horvát
Answer
11 days ago

Ok I check your link and did some test with NetLink and after I executed the example for LibraryLink. In NetLink mmm sound like the output array size should be known because I allocate array before call the function. But if I need an function that return an array size where the size is determinated from the dll function itself it's impraticable.

After I test LibraryLink. with one example and work right. But what's I need is call a DLL that use some c++ code and template. If I tried add a simple class in MyLib.c compiler give me an error. I suppose its set a c compiler. It's possibile use a c++ compiler in LibraryLink that support c++ template? I installed a mingw compiler c++ and set this as default. I use this command to set default compiler.

$CCompiler = {"Compiler" -> 
    CCompilerDriver`MinGWCompiler`MinGWCompiler, 
   "CompilerInstallation" -> "C:\\MinGW32-4.8.2"};
POSTED BY: Alessio Alessio
Answer
11 days ago

LibraryLink is a C interface. This means that it is straightforward to use it either from C or C++.

If you use CreateLibrary from within Mathematica, you should place the source code in a separate file that has a .cpp extension. Do not write the code in a Mathematica string. Put it in a separate file. Then everything will work out of the box.

If you use Windows, I strongly recommend you use the officially supported Microsoft compiler. You can download a compatible version here. Install the command line tools and the Windows SDK. No addition configuration is needed from within Mathematica.

Then start with the basic examples here:

POSTED BY: Szabolcs Horvát
Answer
11 days ago

Hello, thank you very much for help. I follow your guide and now I am able to compile some c++ code. I tried the CreateMatrix example. Now I would like ask you if it's possibile return a list with different number of elements. I need an array vertex {{1,2,3}, {5,4,6}, {3,4,5}, ...} as input parameters and as output an array like this

{{1,2,3},{2,3,4,5,6,7,8},{3,3}}

It's possibile return a similar array to Mathematica? Mathematica type it as "List".

POSTED BY: Alessio Alessio
Answer
10 days ago

You will find the full LibraryLink user guide here:

LibraryLink only has direct support for a few types, mostly full arrays. Ragged arrays (i.e. a general list of lists) is not directly supported. You can use one of these two indirect ways:

  1. Use MathLink from within a LibraryLink function. See the last two examples here, as well as the user guide. "MathLink" has recently been renamed WSTP for silly marketing reasons, which causes all sorts of inconveniences. You will notice that the user guide uses WS-prefixed functions, and the example I linked to uses ML-prefixed ones. The fact is that the WS-prefixed functions do not work in a straightforward way with LibraryLink (despite what the documentation says). Just use the ML-prefixed ones, and forget about "WSTP". MathLink works just fine, even if it is "deprecated" according to the documentation.

    If you go this route, you will need to learn a bit of MathLink programming. I recommend this guide.

  2. Flatten your array into one long vector. Prepends the length of each sublist. Then prepend the number of sublists. Thus you would transform
    {{1,2}, {3,4,5}}

    into

    {2 (* two lists *), 2 (* of length 2 ... *), 3 (* ... and length 3 *), (* { *)1, 2 (* } *), (* { *) 3, 4, 5  (* } *)
    

    Then write a decoder function for it in Mathematica.

    This approach will be much faster for lists with lots of sublists than the MathLink-based method.

    Here's a function to unpack such a format into a ragged array (quickly adapted from something I use, but not tested)

<!-- -->
    (* almost the same as Internal`PartitionRagged *)
    partitionRagged[v_List, l_?VectorQ] := 
     MapThread[Take[v, {#1, #2}] &, 
      Module[{a = Accumulate[l]}, {a - l + 1, a}]]


    unpack[packed_] :=
     With[{len = Round@First[packed]},
      partitionRagged[
       1 + packed[[len + 2 ;; All]],
       Round[packed[[2 ;; len + 1]]]
       ]
      ]
POSTED BY: Szabolcs Horvát
Answer
10 days ago

When I have used LibraryLink, I did all the transfers using MathLink, which is probably slower but much more powerful.

POSTED BY: Frank Kampas
Answer
10 days ago

When I have used LibraryLink, I did all the transfers using MathLink, which is probably slower but much more powerful.

That is very true. With MathLink it is possible to handle any Mathematica expression. With plain LibraryLink, only a few.

But if you only use LinkObject passing in LibraryLink, then is there any point to using LibraryLink at all? It also comes with big disadvantages compared to installable MathLink programs. If your code crashes, it will take down the whole kernel.

(I guess there may be a few reasons: you can use managed library expressions. But those don't get a lot of use due to all the extra work they need. LTemplate does take care of that problem though.)

POSTED BY: Szabolcs Horvát
Answer
10 days ago

I was using LibraryLink and MathLink to transfer data to and from a C program which called Ipopt. I had to transfer the objective function, constraints, and their first and second derivatives as callbacks. ParametricIPOPT in version 11 rendered my code obsolete.

POSTED BY: Frank Kampas
Answer
10 days ago

Hey it's interesting... can you give me some link with example? I would like learn also this method.

POSTED BY: Alessio Alessio
Answer
10 days ago

@Frank Kampas:

Have you seen the "library callback functions"? They should offer much better performance than MathLink. They are only available since v10.0. I have not tried this functionality yet.

POSTED BY: Szabolcs Horvát
Answer
10 days ago

Thanks, I prefer pack solution. I tried in mathematica the function unpack. Have you a pack function to send data from mathematica to c++ side? If I need to send from mathematica to c++ an array like it {{1,2,3},{1,2},{5,6,7,8,9}}. Now I am not able to write some pack function by could use the same way and unpack in c++ side.

POSTED BY: Alessio Alessio
Answer
10 days ago

I do agree with Frank that unless you have special performance requirements, MathLink is a better choice here. It takes a bit of time to learn it, but once you have, you will find it much easier to use than all this error-prone packing/unpacking business. I used this packing technique (for integers, not reals) because I did have special performance requirements. Due to the nature of the data I was working with, the data transfer itself took much longer than the computation.

POSTED BY: Szabolcs Horvát
Answer
10 days ago

I have special performance requirements too and large array to import in mathematica. Every time that I call createLibrary mathematica try to create a new dll and when it exist get a boring error CopyFile::filex: Cannot overwrite existing file C:\Users\ale\AppData\Roaming\Mathematica\SystemFiles\LibraryResources\Windows\myLib.dll.

Do you know how solve this problem? To get a new version of dll for every modification I need to close mathematica delete dll files and execute again the command createLibrary this is very boring and time consuming.

POSTED BY: Alessio Alessio
Answer
7 days ago

This may not work on all platforms, but it works on my Mac with M11.1.1:

Use LibraryUnload on the library before recompiling.

After recompiling, you will need to re-evaluate all the LibraryFunctionLoads one by one.

My LTemplate package makes this workflow slightly easier as you can just use a single LoadTemplate instead of multiple LibraryFunctionLoads. The library still needs to be unloaded before recompiling with LTemplate. This is done using UnloadTemplate.

POSTED BY: Szabolcs Horvát
Answer
7 days ago

thank you very much LibraryUnload resolve. Is it possible to make the dll part of the base mathematica library ?

POSTED BY: Alessio Alessio
Answer
7 days ago

Is it possible to make the dll part of the base mathematica library ?

I am not sure what you mean by this. Can you clarify?

POSTED BY: Szabolcs Horvát
Answer
7 days ago

sorry for my english. I did a library with LibraryLink. It's possible avoid every time I startup mathematica to execute the notebook that compile the dll files and load it to use the imported function?

POSTED BY: Alessio Alessio
Answer
7 days ago

You can compile it once, then create a plain text .m file (a package file) that contains all the LibraryFunctionLoad commands for it. Then you can simply load that file using Get.

POSTED BY: Szabolcs Horvát
Answer
6 days ago

do you mean ?

http://reference.wolfram.com/language/tutorial/SettingUpExternalFunctionsToBeCalledFromTheWolframLanguage.html

that allows a two way exchange over a unix (file/stream) socket called a LiinkObject (which also is used for remote kernels, remote kernels speak the same link language on the socket, per say)

HOWEVER you said memory and that depends. you can copy the data over the link (i'm unsure if your avoiding that so will continue ...)

you could return a unix shared memory location to a mathematica program that can directly read user space memory. if it's CL video card data then that memory is shared.

BETTER: is copy only a filename over the link containing data which is on a memory drive made in shared memory. now your (other side, a notebook) can poke (bytes) of data from the file after opening file without ever the whole file being copied over the link

On a modern system files are cached very well and if used temporarily never are committed to (slow memory or drive), so you really don't need the trouble of insuring file is located on a shared memory drive i think.

In the unix world there is an old saying: "everything is a file"

POSTED BY: John Hendrickson
Answer
6 days ago

Hello, I tried to build a package that load a dll but when I call the package function I get the following error (screenshot with error)

error function package

If I tried to copy the body of package function on notebook and it work.

This is my package file "meshAlgorithms.m"

BeginPackage["meshAlgorithms`"]

AdjVertex::usage = "adjVert=adjVertex[v, f] returns a list adj vertex";
Begin["`Private`"]

partitionRagged[v_List,l_?VectorQ]:=MapThread[Take[v,{#1,#2}]&,Module[{a=Accumulate[l]},{a-l+1,a}]]
unpack[packed_]:=With[{len=Round@First[packed]},partitionRagged[packed[[len+2;;All]],Round[packed[[2;;len+1]]]]]

AdjVertex[v_,f_]:=Module[{l={},lib,vertexAdj},
lib=FindLibrary[ "vertexAdj"];
vertexAdj=LibraryFunctionLoad[lib,"vertexAdj",{{Real,2},{Integer,2}},{Integer,1}];
(unpack[vertexAdj[vdata,idata]])];

End[]
EndPackage[]
POSTED BY: Alessio Alessio
Answer
4 days ago

sorry I wrong to call function vertexAdj[vdata,idata] instead of vertexAdj[v,f]...

POSTED BY: Alessio Alessio
Answer
4 days ago

Group Abstract Group Abstract