MathCompile is a package I am working on as a different approach to code compilation than Compile and FunctionCompile. In my previous post, I talked about some basic aspects of the functionalities provided in version 0.1.0. Since then, there have been a few very important updates that might fundamentally change how we make use of MathCompile. In this post, I will introduce to you these new features.

MathCompile v0.2.3 has recently been released. You can find the PacletInstall command on the release page and execute it in Mathematica to install or update the package. All 301 compilable functions can be found here, and a complete guide to the package can be found here. MathCompile is still at its early stage where a lot of features are to be added and bugs are expected. Any feedback and comments are welcome.

Strings

Since v0.2.0, string can be passed as arguments and the return value from a compiled function. String manipulating and matching functions that are commonly used are supported as well. MathCompile implements short string optimization for strings of sizes less than about 30 bytes, which improves the efficiency of handling short strings.

As for string matching, MathCompile adopt a similar approach to Mathematica. The compiled function translate each string expression into a regular expression at runtime and invoke PCRE2 library to perform string matching and replacing. StringPattern`PatternConvert can be used within a compiled function to check the regular expression to which the string pattern is translated.

Import and export

MathCompile now supports some basic import and export of binary and text files. They may contain integers, real numbers, or strings. The performance of importing and exporting of may be improved by compilation: for a 1000x1000 matrix of real numbers in TSV format, there is a 2x improvement in importing and a 8x (Ubuntu, GCC) or 20x (Windows, MSVC) improment in exporting.

As for now, Import support three formats for text files: "TSV", "CSV", and "Table", and the data type can be integral or floating-point. For example, the following function imports a CSV file as an array of single-precision floating-point numbers (Real32),

CompileToBinary@Function[{Typed[file,String]},Import[file,{"CSV","Real32"}]]

Import can also be used with binary files, where the format should be specified as "Binary", e.g.

CompileToBinary@Function[{Typed[file,String]},Import[file,{"Binary","Real32"}]]

Note that the data type should always be specified when using Import, but is not required by Export because MathCompile would have known the type of the exported expression during compilation.

In addition to Import/Export, MathCompile supports input and output streams. You can use functions like OpenRead and OpenWrite to open a file stream and read from or write to the stream using ReadLine, WriteLine, etc. StreamPosition and SetStreamPosition can be used to get and set the position in the stream.

Linear algebra

MathCompile supports linear algebra functions by calling Eigen library, such as LinearSolve, LeastSquares, and various matrix decompositions. MathCompile provides BLAS and LAPACK interfaces so that you can link extern BLAS and LAPACK libraries to replace Eigen (see here for more information).

Note that matrix decompositions may yield a number of arrays with different types and dimensions. MathCompile supports assigning multiple variables using a single Set, and it can be used to do initializations. For example,

Module[{lu,p,c},{lu,p,c}=LUDecomposition[{{1,1},{1,0}}];]

initializes the variables lu, p, and c with three elements of the LU decomposition of the matrix.

Calling kernel functions / inline C++

Since v0.2.2, MathCompile supports calling kernel functions in compiled functions. This features enables you to make a function call that is going to be executed in the Wolfram Kernel, like KernelFunction but with a different representation. A common usage of this feature is to embed a function that is not compilable, e.g.

CompileToBinary@Function[{Typed[x,Real]},Extern[AiryAi,Typed[Real]][x]]

where Extern[AiryAi,Typed[Real]] represents a kernel function AiryAi that returns a real number (Typed[Real]). Another use case is to pass a parameter to a function that is already compiled, e.g.

f = CompileToBinary@Function[{Typed[x,Integer]},x+Extern[gety,Typed[Integer]][]];

By defining gety, you can change the behavior of this function without recompiling it:

gety[] := 5;
f[10]  (* returns 15 *)
gety[] := 8;
f[10]  (* returns 18 *)

MathCompile also supports inlining C++ code, which is specified by the function CXX. For instance,

CompileToCode@Function[{Typed[x, Real]}, CXX["std::floor"][x]]

applies a C++ function std::floor on variable x. To access a variable in Wolfram Language from the inlined C++ code, you need to wrap the name of the variable by back-tics "`", e.g.

CompileToCode@Function[{},Module[{x=1}, CXX["`x`+=5"];x]]]

MathCompile will replace the variables with their correct names during compilation.