I am a Data Scientist working in a startup in Brazil. Recently I was working with Social Networks in Python using networkx library to analyse posts from a given hashtag for our client. However, I was completely aware of the additional functionalities of Wolfram Mathematica. Take the following picture as an example, done with networkx, 100,000 people:
It's an interesting perspective, but adds little value to generate strategic insights for our clients, given that we need to zoom it to analyze details of this network. In my previous experience of Wolfram Mathematica, I was able to use some reasoning coming from cellular automata interactions to map, track people of interest (blue circle), and watch the evolution of mood (color), number of connections (size of circle) in a social network:
Besides, one can use Mathematica's features of finding communities, coloring them and even highlight people of interest when passing the mouse over the social network and also speaking their names:
More than that, you can highlight communities and check the number of connections of each individual:
So, it's clear that Wolfram Mathematica is way ahead of networkx. However, I was dealing with a drawback: if you choose to plot more than 10,000 connections in Mathematica, that can take some time. And time is what you don't have in a startup. So, I started wondering how I could use GPUs with Wolfram Mathematica. One way it's to use webMathematica, installing Java and Apache Tomcat in a cloud instance. However, I was told one can also use Wolfram Client Python library and run Mathematica in a Python notebook.
So, I will present how I was able to run Mathematica inside a Jupyter notebook located in a Google Cloud instance with 8 V100 GPUs. My co-worker, Gustavo Gouvea, also put efforts in this solution.
First of all, go to Google Cloud Platform (GCP) Compute Engine and select Create Instance in a given region. Choose your CPUs, Memory, GPUs and regarding the boot disk, I used Debian GNU/Linux with Anaconda, PyTorch and CUDA already installed, as we also work with Deep Learning and NLP.
After that, you need to configure Jupyter in order to be able to open it in the local browser, by doing the following:
ipython
from IPython.lib import passwd
passwd()
Now that you created the password, you are going to save "sha1:49b8799c22..." Then you will edit Jupyter configuration file (use sudo or chmod -R 777 /home/anaconda3):
sudo vi ~/.jupyter/jupyter_notebook_config.py
Add (type "i") these line of code:
c=get_config()
c.NotebookApp.password = paste your sha1 here
c.IPKernelApp.pylab = 'inline'
c.NotebookApp.ip = "*" or 0.0.0.0
c.NotebookApp.open_browser = False
Go to GCP VPC and create a static IP for your instance, SSH into it.
So, to start Jupyter notebook you will need to run:
jupyter notebook
Now you will access Jupyter in the following address:
http://http://55.100.30.200:8888/
That will allow you to run Python notebooks in Jupyter. Now Mathematica. Get the mathematica .sh file:
sudo wget link_to_mathematica.sh
To install Mathematica run in another SSH session (terminal window):
sudo bash link_to_mathematica.sh
Ok, you installed Mathematica. Now run:
math
This command will ask for your Activation key and Password. After providing them, Mathematica notebook starts in the command line
Important: You must run Mathematica ("math") before you open Jupyter Notebook.
Now that you've opened Mathematica in the command line with one SSH and after that Jupyter Notebook in other SSH, access the web address, port 8888, create a new Python 3 notebook and install wolframclient library:
Now you can run your Mathematica scripts with backup of how many GPUs you need:
from wolframclient.language import wl as w1
from wolframclient.language import wlexpr
from wolframclient.evaluation import WolframLanguageSession
session = WolframLanguageSession('/usr/local/Wolfram/Mathematica/12.0/Executables/MathKernel')
import numpy as np
import imageio
import matplotlib.pyplot as plt
from PIL import Image
In bird's eye:
The only point is that you won't have access to all Mathematica functionalities, as Manipulate, Mouse Over and the display of some images when output is truncated. My guess is that init.m config file must be adjusted to increase output size before truncation (in the command line) :