I have recently added LibraryLink Support on mathematica++. It is still in develop branch. It'll take a few days before releasing it to the master branch.

It already have features to stream WSPut/MLPut calls streamed through the mlink or wslink.

shell << Values(FindRoot(ArcTan(1000 * Cos(x)), List(x, 1, 2),  Rule("Method") = method));

And it can cast tokens fetched from mathematica into generic stl container, like std::vector, std::pair, boost::tuple etc..

Exploiting these two features the aim of the Library Link support is to wrap only the difficult parts while letting the programmer focus on the implementation functions. I started the development process with the following 4 concerns in mind.

• Unharmed Skeleton The boilerplate of 4 functions, initialize, uninitialize, version and the actual function to be exported is left intact with the exact signature, so that the developer can easily follow up after reading the LibraryLink documentation on Mathematica website. It also doesn't restrict the developer to code whatever he/she could with plain C API.
• C++ Environment The developer should be able to develop and unit test the module only using C++ based environment without using Mathematica fronted. However the developer is not barred from using the frontend in the middle of the development process. The concern is a C++ developer should feel home like while developing a module.
• Support Generics standard C++ generic containers should be supported.
• Extensible The library should be extensible to support 3rd party C++ libraries. (Eigen Matrix is already supported)

Following is an example how link based Library development is abstracted. The native_link can still be used with the C API. The resolver redirects calls to different impl functions either based on heads or based on number of arguments.

EXTERN_C DLLEXPORT int SomeFunctionX(WolframLibraryData libData, WMK_LINK native_link){
   mathematica::wtransport shell(libData, native_link);

   try{
        mathematica::resolver resolver(shell);
        resolver, overload(&some_function_impl_geo, shell) = {"GeoPosition", "GeoPosition"}
                , overload(&some_function_impl_complex) = {"Complex", "Complex"}
                , overload(&some_function_impl_binary)
                , overload(&some_function_impl_unary);
        return resolver.resolve();
    }catch(...){
        return shell.pass();
    }
    return 0;
}

Following are the signatures of the impl functions in the above example. overload checks the function argument types and cast accordingly. This can can handle user defined types like the point_2d in the example through different specialized serialization policies.

double some_function_impl_geo(mathematica::transport& shell, point_2d<double> p1, point_2d<double> p2);
double some_function_impl_complex(std::complex<double> p1, std::complex<double> p2)
int some_function_impl_binary(double x, double y);
mathematica::m some_function_impl_unary(double x);

The shell (first argument in the first overload) comes from libData and can be used to do computations with mathematica. In the example some_function_impl_geo converts point_2d to GeoPosition and then passes to GeoDistance as arguments.

double some_function_impl_geo(mathematica::transport& shell, point_2d<double> p1, point_2d<double> p2){    
    std::string unit("Kilometers");
    double res;
    shell << QuantityMagnitude(GeoDistance(p1, p2, Rule("!UnitSystem") = unit));
    shell >> res;
    return res;
}

The returned output is sent back to the frontend. The last overload in the example returns a mathematica expression.

As previously mentioned no flavours are abandoned. The MArgument based style is abstracted in mathematica::mtransport. mtransport doesn't support overloads, type of its input is expected to be fixed with LibraryFunctionLoad. Input output with tensor of any type, rank and dimension is supported. Type of nesting level of std::vector determines the expected type and rank of the tensor. Currently only std::vector is supported, however there are plans to support eigen and other containers.

args = shell is used to capture the input arguments, whereas shell = ret is used to get the returned value.

EXTERN_C DLLEXPORT int SomeFunctionMXT(WolframLibraryData libData, mint argc, MArgument* argv, MArgument res){
    mathematica::mtransport shell(libData, argc, argv, res);
    typedef std::vector<std::vector<double>> matrix_type;
    try{
        boost::tuple<matrix_type, matrix_type> args = shell; // fetch input
        matrix_type matl, matr, mato;   // declare variables to hold the input and output
        boost::tie(matl, matr) = args;  // tie tuple to variables
        shell << Dot(matl, matr);       // To lazy to write matrix multiplication code in C++ so using mathematica instead
                                  // This converts matl and matr to the corresponding List[..] expression
        shell >> mato;            // retrieve multiplication result as List[...] and cast to matrix_type
        shell = mato;             // return the result
    }catch(...){
        return shell.pass();
    }
    return LIBRARY_NO_ERROR;
}

shell.pass() passes the exceptions to the frontend and aborts.

I'd be glad to hear reviews and comments from others.

I just pushed some changes to use The PutArray functions is the input is a nesting of std::vector<std::vector<...>> and that vector forms a matrix or higher dimensional matrix or tensor of proper shape.

typedef std::vector<std::vector<std::vector<int>>> ivv_type; // 2 x 2 x 3
ivv_type matrix;
std::vector<int> row11 = {111, 112, 113};
std::vector<int> row12 = {121, 122, 123};
std::vector<int> row21 = {211, 212, 213};
std::vector<int> row22 = {221, 222, 223};

std::vector<std::vector<int>> row1;
std::vector<std::vector<int>> row2;

row1.push_back(row11);
row1.push_back(row12);

row2.push_back(row21);
row2.push_back(row22);

matrix.push_back(row1);
matrix.push_back(row2);

value res;

shell.enable(mathematica::bulk_io); // ENABLE this feature
shell << Total(Flatten(matrix));
shell >> res;

BOOST_CHECK(*res == 2004);

The above will use MLPutInteger32Array/ WSPutInteger32Array

These two changes are currently in develop branch. I'll have some more tests and then I'll put them in master branch.

After that I'll use Armadillo Eigen etc.. as optional dependencies.

I can detect a packed array but I cannot detect whether all elements in that are are int64 or int32 or double or string because List is not uniform. So I wrap it in a variant type and fetch one by one.

Aha I see. Yes I was talking about detecting a List. I'll have a look into packed array

What I was trying to say was that it would be nice to have zero runtime overhead---if at all possible. As I understand from your answer, this library will effectively translate the convenience syntax to WS... calls at runtime, not at compile time. There is extra code that runs when using this library, that would be avoided if using WS... functions directly.

I am not sure about the feasibility of all compile time solution.I may work to some extent but there are scenarios that are difficult to tackle this way. Given a very simple use case

List[1, 2, 3, 4] // mathematica syntax

A similar C++ syntax List<1, 2, 3> which build the WS call chain on compile time, but it is not possible to pass string in this way.

A syntax like a = List(1, 2, 3, "hallo") can builds the call chain at compile time. But a scenario like the following

a = List(); // empty List initiated
for (auto x: l){
    a.arg(x);
}

expecting to build a List (not append) at the end of the loop requires a runtime storage for the call chain. Otherwise if it entirely built on templates some more magic like b = arg(a, x) can be used to abandon a and generate a new b. i.e. once an object is created it wont be mutable. Whether it is a good design or bad design is a matter of debate and depends on use case. I am not very much sure about such feasibility. And I am afraid that things may turn ugly.

I thought about this for quite a while and decided to store and populate it on runtime with delayed callbacks. Even if I generate the call chain on compile time there will be some wrapper around the WSTP functions. That part is unavoidable. The only extra overhead is the runtime storage of the call chain. Each delayed call ftor requires few words. So for the second part of your question, yes there is an extra code that is running but I don't consider it as a bottleneck.

This is however very inefficient when working with numerical arrays, a very common task in scientific programming

Yes then there needs to be case by case exception in the fetch call. However the problem is while fetching the library knows that it is fetching a List. But the library doesn't know that it is fetching a list of all 64 bit integers. While making Put calls the exceptional rules may still work but for Get calls I am not sure about its feasibility.

The problem is that if one decides to write a program that runs in a separate process and communicates with the kernel through MathLink, one does have the option to choose either MathLink (ML prefix) or WSTP (WS prefix), or switch between the two with the preprocessor technique you mentioned. However, when using LibraryLink, which is usually a better option than installable MathLink programs, one must use MathLink (ML-prefix functions). Thus basing such a library on the wstp.h header will limit its usability (no LibraryLink). Basing it on mathlink.h wouldn't.

I don't think there is any benefit criticizing wolfarm's decision in this regard this is what is happening for few years. It doesn't seem to go in the ML direction in future. So I targeted with WSTP only. However I kept all low level (WS) calls in few places so that it never turns hectic to change from WS to ML. Currently I am busy for an week. I'll see if I can push that change in few days.

Thanks for the suggestion. I am not sure about the first one but the second one is easy to accomplish. Although PutReal64Array is feasible I am doubtful about GetReal64Array

Yes it uses template magic in the expected way ...

... Actually I am storing chain of delayed callbacks that are fired in order.

What I was trying to say was that it would be nice to have zero runtime overhead---if at all possible. As I understand from your answer, this library will effectively translate the convenience syntax to WS... calls at runtime, not at compile time. There is extra code that runs when using this library, that would be avoided if using WS... functions directly.

My question was: do you have any idea of how great this overhead is?

While implementing I avoided functions like PutReal64List Instead I have gone in a more generic pathway. creating a list first and then Put numbers one by one.

This is however very inefficient when working with numerical arrays, a very common task in scientific programming ...

It uses wstp.h only, and only WS* functions. However user code does not need to include that header. A good amount of WS functions and ML functions share their name so it is more or less the same (one macro away). It is trivial to keep a macro that switches between WS functions and ML functions.

MathLink and WSTP are in fact exactly the same, not more or less the same. "Replacing" MathLink with WSTP was an entirely pointless marketing move that is causing real inconvenience and extra work for people who work with this technology.

The problem is that if one decides to write a program that runs in a separate process and communicates with the kernel through MathLink, one does have the option to choose either MathLink (ML prefix) or WSTP (WS prefix), or switch between the two with the preprocessor technique you mentioned. However, when using LibraryLink, which is usually a better option than installable MathLink programs, one must use MathLink (ML-prefix functions). Thus basing such a library on the wstp.h header will limit its usability (no LibraryLink). Basing it on mathlink.h wouldn't.

Thank you for the very first review. I am glad to know that it worked nicely.