Community RSS Feed
https://community.wolfram.com
RSS Feed for Wolfram Community showing any discussions in tag External Programs and Systems sorted by activeThoughts on a Python interface, and why ExternalEvaluate is just not enough
https://community.wolfram.com/groups/-/m/t/1185247
`ExternalEvaluate`, introduced in M11.2, is a nice initiative. It enables limited communication with multiple languages, including Python, and appears to be designed to be relatively easily extensible (see ``ExternalEvaluate`AddHeuristic`` if you want to investigate, though I wouldn't invest in this until it becomes documented).
**My great fear, however, is that with `ExternalEvaluate` Wolfram will consider the question of a Python interface settled.**
This would be a big mistake. A *general* framework, like `ExternalEvaluate`, that aims to work with *any* language and relies on passing code (contained in a string) to an evaluator and getting JSON back, will never be fast enough or flexible enough for *practical scientific computing*.
Consider a task as simple as computing the inverse of a $100\times100$ Mathematica matrix using Python (using [`numpy.linalg.inv`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.inv.html)).
I challenge people to implement this with `ExternalEvaluate`. It's not possible to do it *in a practically useful way*. The matrix has to be sent *as code*, and piecing together code from strings just can't replace structured communication. The result will need to be received as something encodable to JSON. This has terrible performance due to multiple conversions, and even risks losing numerical precision.
Just sending and receiving a tiny list of 10000 integers takes half a second (!)
In[6]:= ExternalEvaluate[py, "range(10000)"]; // AbsoluteTiming
Out[6]= {0.52292, Null}
Since I am primarily interested in scientific and numerical computing (as I believe most M users are), I simply won't use `ExternalEvaluate` much, as it's not suitable for this purpose. What if we need to do a [mesh transformation](https://mathematica.stackexchange.com/q/155484/12) that Mathematica can't currently handle, but there's a Python package for it? It's exactly the kind of problem I am looking to apply Python for. I have in fact done mesh transformations using MATLAB toolboxes directly from within Mathematica, using [MATLink][1], while doing the rest of the processing in Mathematica. But I couldn't do this with ExternalEvaluate/Python in a reasonable way.
In 2017, any scientific computing system *needs* to have a Python interface to be taken seriously. [MATLAB has one][2], and it *is* practically usable for numerical/scientific problems.
----
## A Python interface
I envision a Python interface which works like this:
- The MathLink/WSTP API is exposed to Python, and serves as the basis of the system. MathLink is good at transferring large numerical arrays efficiently.
- Fundamental data types (lists, dictionaries, bignums, etc.) as well as datatypes critical for numerical computing (numpy arrays) can be transferred *efficiently* and *bidirectionally*. Numpy arrays in particular must translate to/from packed arrays in Mathematica with the lowest possible overhead.
- Python functions can be set up to be called from within Mathematica, with automatic argument translation and return type translation. E.g.,
PyFun["myfun"][ (* myfun is a function defined in Python *)
{1,2,3} (* a list *),
PyNum[{1,2,3}] (* cast to numpy array, since the interpretation of {1,2,3} is ambiguous *),
PySet[{1,2,3}] (* cast to a set *)
]
- The system should be user-extensible to add translations for new datatypes, e.g. a Python class that is needed frequently for some application.
- The primary mode of operation should be that Python is run as a slave (subprocess) of Mathematica. But there should be a second mode of operation where both Mathematica and Python are being used interactively, and they are able to send/receive structured data to/from each other on demand.
- As a bonus: Python can also call back to Mathematica, so e.g. we can use a numerical optimizer available in Python to find the minimum of a function defined in Mathematica
- An interface whose primary purpose is to call Mathematica from Python is a different topic, but can be built on the same data translation framework described above.
The development of such an interface should be driven by real use cases. Ideally, Wolfram should talk to users who use Mathematica for more than fun and games, and do scientific computing as part of their daily work, with multiple tools (not just M). Start with a number of realistic problems, and make sure the interface can help in solving them. As a non-trivial test case for the datatype-extension framework, make sure people can set up auto-translation for [SymPy objects][3], or a [Pandas dataframe][4], or a [networkx graph][5]. Run `FindMinimum` on a Python function and make sure it performs well. (In a practical scenario this could be a function implementing a physics simulation rather than a simple formula.) As a performance stress test, run `Plot3D` (which triggers a very high number of evaluations) on a Python function. Performance and usability problems will be exposed by such testing early, and then the interface can be *designed* in such a way as to make these problems at least solvable (if not immediately solved in the first version). I do not believe that they are solvable with the `ExternalEvaluate` design.
Of course, this is not the only possible design for an interface. J/Link works differently: it has handles to Java-side objects. But it also has a different goal. Based on my experience with MATLink and RLink, I believe that *for practical scientific/numerical computing*, the right approach is what I outlined above, and that the performance of data structre translation is critical.
----
## ExternalEvaluate
Don't get me wrong, I do think that the `ExternalEvaluate` framework is a very useful initiative, and it has its place. I am saying this because I looked at its source code and it appears to be easily extensible. R has zeromq and JSON capabilities, and it looks like one could set it up to work with `ExternalEvaluate` in a day or so. So does Perl, anyone want to give it a try? `ExternalEvaluate` is great because it is simple to use and works (or can be made to work) with just about any interpreted language that speaks JSON and zeromq. But it is also, in essence, a quick and dirty hack (that's extensible in a quick and dirty way), and won't be able to scale to the types of problems I mentioned above.
----
## MathLink/WSTP
Let me finally say a few words about why MathLink/WSTP are critical for Mathematica, and what should be improved about them.
I believe that any serious interface should be built on top of MathLink. Since Mathematica already has a good interface capable of inter-process communication, that is designed to work well with Mathematica, and designed to handle numerical and symbolic data efficiently, use it!!
Two things are missing:
- Better documentation and example programs, so more people will learn MathLink
- If the MathLink library (not Mathematica!) were open source, people would be able to use it to link to libraries [which are licensed under the GPL][6]. Even a separate open source implementation that only supports shared memory passing would be sufficient—no need to publish the currently used code in full. Many scientific libraries are licensed under the GPL, often without their authors even realizing that they are practically preventing them from being used from closed source systems like Mathematica (due to the need to link to the MathLink libraries). To be precise, GPL licensed code *can* be linked with Mathematica, but the result cannot be shared with anyone. I have personally requested the author of a certain library to grant an exception for linking to Mathematica, and they did not grant it. Even worse, I am not sure they understood the issue. The authors of other libraries *cannot* grant such a permission because they themselves are using yet other GPL's libraries.
[MathLink already has a more permissive license than Mathematica.][7] Why not go all the way and publish an open source implementation?
I am hoping that Wolfram will fix these two problems, and encourage people to create MathLink-based interfaces to other systems. (However, I also hope that Wolfram will create a high-quality Python link themselves instead of relying on the community.)
I have talked about the potential of Mathematica as a glue-language at some Wolfram events in France, and I believe that the capability to interface external libraries/systems easily is critical for Mathematica's future, and so is a healthy third-party package ecosystem.
[1]: http://matlink.org/
[2]: https://www.mathworks.com/help/matlab/matlab-engine-for-python.html
[3]: http://www.sympy.org/
[4]: http://pandas.pydata.org/
[5]: https://networkx.github.io/
[6]: https://en.wikipedia.org/wiki/Copyleft
[7]: https://www.wolfram.com/legal/agreements/mathlink.htmlSzabolcs Horvát2017-09-15T12:33:04ZMathematica 12.0 on Linux issue with CUDA
https://community.wolfram.com/groups/-/m/t/1661787
I installed Mathematica 12.0 on Ubuntu Linux 18.10 and activated cuda paclet 12.0.287.
This simple code:
Needs["CUDALink`"]
array = Table[i + j, {i, 1000}, {j, 1000}];
Timing[CUDADot[array, array]]
gives the error:
/usr/local/Wolfram/Mathematica/12.0/SystemFiles/Kernel/Binaries/Linux-x86-64/WolframKernel: symbol lookup error: /home/bert/.Mathematica/Paclets/Repository/CUDAResources-Lin64-12.0.287/LibraryResources/Linux-x86-64/libCUDALink_Double.so: undefined symbol: cublasCreate_v2
I have nVIDIA driver 418.56 supporting CUDA 10.1 (while the paclet is based on 10.0).
I start Mathematica with following script:
export NVIDIA_DRIVER_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/libnvidia-tls.so.418.56
export CUDA_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/libcuda.so
mathematica &
Am I missing something?Bert Aerts2019-04-16T20:22:53ZDefine a new output format based on an existing output format, e.g. Fortran
https://community.wolfram.com/groups/-/m/t/1663445
Hallo *.*!
I would like to define a new output format based on an existing output format (e.g. FortranForm), and I'll call it "PascalForm".
My first idea ...
1.
Format[x_, PascalForm] := Format[x, FortranForm]
2.
Format[x_ ^ y_, PascalForm] := StringForm["power(\`\`,\`\`)", x, y]
fails twice.
to 1. with ...
Format::nosym: x_ does not contain a symbol to attach a rule to. >>
to 2. with ...
SetDelayed::write: Tag Power in y_
(x_) is Protected. >>
Do you have an idea that could work?
Or does someone know the suitable documents to my task?
Or is it the depths of internet already a difficult to find solution for me?
Thank you for your help.
Greetings
BerndBernd Rueffer2019-04-18T10:18:12ZNeural Network on Cortex-M ARM processors with Mathematica
https://community.wolfram.com/groups/-/m/t/1661629
## Introduction ##
Disclaimer : I am an ARM employee but this is a personal work.
I wanted to understand how to convert a Neural network from Mathematica to use it on a Cortex-M with the CMSIS-NN library. CMSIS-NN is a free ARM library containing a few optimized functions for Neural networks on embedded systems (convolutional layers and fully connected).
There are a few demos (CIFAR and Keyword spotting) running on Cortex-M. There were generated either from Caffe framework or with TensorFlow Lite.
I wanted to do the same from Mathematica and also understand a bit more the CMSIS-NN library. So, I attempted to reproduce a keyword spotting example.
## The Network ##
The network is quite simple but for an embedded system we cannot use something too complex.
First the audio is going through a MFCC step:
audioEnc =
NetEncoder[{"AudioMFCC", "WindowSize" -> 4*160, "Offset" -> 2*160,
"NumberOfCoefficients" -> 10, "TargetLength" -> 49,
SampleRate -> 16000}]
The network is standard : a few convolutional layers followed by a few fully connected layers:
kwsModel = NetChain[{
ReplicateLayer[1],
ConvolutionLayer[channels[[1]], {10, 4}],
ElementwiseLayer[Ramp],
ConvolutionLayer[channels[[2]], {10, 4},
"Input" -> {channels[[1]], 40, 7}, "Stride" -> {2, 1}],
ElementwiseLayer[Ramp],
LinearLayer[58],
ElementwiseLayer[Ramp],
LinearLayer[128],
ElementwiseLayer[Ramp],
LinearLayer[Length[wantedClass]],
SoftmaxLayer[]
}, "Input" -> audioEnc, "Output" -> classDec]
At the output, there is a NetDecoder which is converting the output into 3 classes. I am trying to detect the word "backward", "yes", "no".
The test patterns are coming from the TensorFlow keyword spotting example (link in attached notebook). But my network is different.
## Problems to solve ##
There are 2 problems to solve to be able to convert this network for CMSIS-NN.
First problem : the library is using a different convention for the tensors which means that the weights have to be reordered before being used by CMSIS-NN. Since it is not too difficult to do with Mathematica, I won't detail it here.
Second problem : CMSIS-NN is using fixed point arithmetic (Q15 or Q7). But Mathematica is using float.
Two limitations of Mathematica : there are no quantization operators so we cannot learn the network with the quantization effects. It is not a major issue but we can expect that the quantized network will be less good than if we had trained directly with quantization effects.
Second limitation : During training Mathematica is not keeping track of the statistics of the intermediate values (input and output of layers). But to convert the float into fixed point we need to know some statistics about the dynamic of those values. Once the dynamic is known, the quantization is controlled with parameters of the CMSIS-NN layers : shift values for the weight and bias.
So, to get those statistics I am just applying each layer of the trained network one after another and keeping track of the input / output. I do this on all training patterns. By luck embedded networks are small so even if it is slow to do this, it is not too slow.
I get beautiful histograms (log scale) which are used as a basis to choose how to quantize the values. A simple strategy is to just use the min and max values.
![Histograms][1]
## Code generation ##
Once I have statistics for the dynamics of the values, I can generate C code for CMSIS-NN and C arrays containing the quantized values.
Since quantization has an effect on the performance of the network, I want to be able to test the result easily. So, I have customized CMSIS-NN to be able to run it from Mathematica. The C code generated by the Notebook can be compiled and used with Mathlink.
Like that I can compare the behavior of the original network and the CMSIS-NN quantized one.
Here is an example:
![UI to play with both networks (Mathematica and CMSIS-NN)][2]
## To use the notebook ##
The steps to convert a network are:
- Train a network
- Compute statistics on the network intermediate values
netStats = ComputeAllFiles[result, audioEnc, trainingFiles, SumStat] ;
result is the trained network.
audioEnc is the MFCC
trainingFiles are the training files.
SumStat is the strategy used for the statistics. Here we just get a summary statistics : just min/max
- Quantize the network and generate C code
mfcc = audioEnc[AudioResample[SpeechSynthesize["backward"], 16000]];
quantizedNetwork1 = CorrectedFormats[result, netStats, 15, 0];
quantizedNetwork = <| "w" -> quantizedNetwork1["w"],
"net" -> Drop[quantizedNetwork1["net"], 1]|>;
TestPatterns[NetDrop[result, 1],
NetExtract[result, 1][mfcc], quantizedNetwork];
CompileNetwork[
NetDrop[result, 1], NetExtract[result, 1][mfcc],
result[mfcc, None], quantizedNetwork]
TestCode[NetDrop[result, 1], quantizedNetwork];
In this example NetDrop is just dropping the first ReplicateLayer. It does not exist in CMSIS-NN and it is used here just to adapt the tensor shape at the input of the network.
The second and third arguments of CompileNetwork are input and output of the network on one test pattern. It is used only when debugging the network.
mfcc is the input pattern (mfcc of some audio pattern).
- Compiling the generated code in ctests using the provided Makefile
- Linking the executable and start using it
link = Install[
FileNameJoin[{NotebookDirectory[], "ctests", "cmsisnn.exe"}]];
cmsiskws[s_] :=
classDec[CMSISNN[
QuantizeData[15, quantizedNetwork["net"][[1, 2]],
Transpose[audioEnc[s] // ReplicateLayer[1] , {3, 2, 1}] //
Flatten]]];
The cmsiskws is a convenience function. It is quantizing the input data using the format computed during quantization of the network.
Then it is computing the MFCC of the audio, reordering the data (Transpose) to use the same convention as CMSIS-NN. Then the CMSISNN function is called on the result.
cmsiskws[AudioResample[SpeechSynthesize["yes"], 16000]]
We can now test that this C code can recognize the word "yes".
The same notebook and same principles were used on CIFAR example.
I can't include the zip containing the C sources to this post. It is not accepted. Without those C sources you won't be able to reproduce the results of this post.
Any idea how
[1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Pict2.png&userId=89693
[2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Pict1.png&userId=89693Christophe Favergeon2019-04-16T08:59:54Z