Message Boards Message Boards

Has anyone gotten ExternalEvaluate to work with Anaconda Python on OS X?


Has anyone gotten ExternalEvaluate to work with Anaconda Python on OS X? It used to work for me in the prerelease, but it's not working in the final release.

In[2]:= py = StartExternalSession["Python"]
Out[2]= ExternalSessionObject["3dac9d41-1400-48be-83ea-9b6012316ff4"]

In[3]:= ExternalEvaluate[py, "1+1"]

During evaluation of In[3]:= Import::nopythonevals: No Python external evaluator found. Use RegisterExternalEvaluator to register an external evaluator.
During evaluation of In[3]:= ExternalEvaluate::interpFail: The result 2 failed to be interpreted as a WL expression. Use "ReturnType"->"String" instead.

Out[3]= $Failed

enter image description here

Notice that Python did return the result 2. What fails is only the translation of this result to Mathematica.

I would like to know if this is broken for everyone or if my system is misconfigured.

After this problem is solved, I would also like to know if it is possible to use different virtualenvs. Can I have multiple external evaluators registered, each corresponding to a specific virtualenv? Update: It seems that virtualenv creates a copy of the python interpreter, thus passing this to RegisterExternalEvaluator should work. But I can't try until ExternalEvaluate starts working on my system ...

POSTED BY: Szabolcs Horvát
6 months ago

I tracked down what is happening here.

The Python expression import functionality seems to rely on a working Python evaluator (see ExternalEvaluate`ImportExport`Symbols`importPythonString).

However, it does not retrieve an evaluator from the list of registered evaluators. Instead, it tries to use various heuristics to find one on the system (see ExternalEvaluate`Private`findLangInstall).

This fails on my system, possibly because I did not install pyzmq for the system's Python. I do not want to modify the OS's Python because I use a private installation (Anaconda) instead. This one already has pyzmq installed, as required.

Why won't Import[..., "PythonExpression"] use the evaluator I registered then? Why does it try to look for an evaluator on its own (and fail)? This seems like a bug.

POSTED BY: Szabolcs Horvát
6 months ago

For those, who like me, don't want to modify their system python (even by installing virtualenv), here's a hack to get things working:

First, make sure that you have successfully registered an external evaluator for Python. FindExternalEvaluators[] should return it.

Then, we trigger loading the ExternalEvaluate framework (in case it hasn't been loaded yet):

In[1]:= ExternalEvaluate
Out[1]= ExternalEvaluate

We manually start up a session with a string return type, and assign it to appropriate the internal variable

In[2]:= ExternalEvaluate`ImportExport`Private`$ImporterPythonSession = StartExternalSession["Python" -> "String"]
Out[2]= ExternalSessionObject["9a23363e-5403-442d-8fbd-39f9aff7dc48"]

And now it works!

In[3]:= py = StartExternalSession["Python"]
Out[3]= ExternalSessionObject["5022a77c-eedb-4d39-bbb3-c8b6a644ecdf"]

In[4]:= ExternalEvaluate[py, "1+1"]
Out[4]= 2

Note: I posted this solution on the Mathematica StackExchange, where it is more likely to stay maintained than here (say, if a paclet update or 11.2.1 fixes this problem).

POSTED BY: Szabolcs Horvát
6 months ago

I tried to register a Python executable installed in a virtualenv with RegisterExternalEvaluator but it didn't seem to work:

In[162]:= virtualenvPython="/Users/meng/python_virtualenvs/virtualenv-test/bin/python2.7";
Out[162]= <|"ExitCode" -> 0, "StandardOutput" -> "", 
 "StandardError" -> "Python 2.7.13
In[161]:= RegisterExternalEvaluator["Python",virtualenvPython]
During evaluation of In[161]:= RegisterExternalEvaluator::invalid: -- Message text not found -- (ExternalEvaluate`Private`reason)
Out[161]= $Failed
6 months ago

I didn't comment on virtualenv further because when I tried it, it worked for me. It is necessary to have pyzmq installed in that virtualenv though. Did you make sure it's installed?

POSTED BY: Szabolcs Horvát
6 months ago

Group Abstract Group Abstract