Message Boards Message Boards

Parallel Mathematica Environment on the RaspberryPi using OOP

GROUPS:

My project, Parallel Mathematica Environment on the RaspberryPi using OOP, is a sample application of Object Oriented Programming for the Mathematica cluster computing, implemented with a Mac and three RaspberryPi Zero connected with a USB hub and three USB cables.

Basic idea is to deploy a constructed instance image to calculating servers (RaspberryPi) and send messages to the instance. OOP on the Mathematica is already developed and shown in this community, and further detail is shown on slidesshare titled of "OOP for Mathematica." enter image description here


Preparing for RaspberryPi Zero is as follows using SSH connection from a Mac,

  • naming each Zero as raspberypi,raspberrypi1,raspberrypi2,...
  • set the server program "init" to each RaspberryPi, init is,

    $ cat init
    While[True,
    Run[“nc -l 8000>input”];
    temp=ReleaseHold[<<input];
    temp >>output;
    Run[“nc your-mac-hostname.local 8002<output”]
    ]
    

where, socket numbers must be identical.

  • Run Mathematica manually, and wait the booting Mathematica up.

    $ wolfram <init&
    

Checking each RaspberryPi is useful as,

$ nc -l 8002 >output|nc raspberrypi.local 8000 <<EOF
> 10!
> EOF
$ cat output
3628800

Cluster controller program on a Mac is,

  • set directory

    SetDirectory[NotebookDirectory[]];
    
  • setup socket communication process

    com1="nc -l 8002 >output1 |nc raspberrypi.local 8000 <input1";
    com2="nc -l 9002 >output2 |nc raspberrypi1.local 9000 <input2";
    com3="nc -l 9502 >output3 |nc raspberrypi2.local 9500 <input3";
    
  • set object property

    obj={
       <|"name"->node1,"comm"->com1,"in"->"input1","out"->"output1","p"->{2000,3500}|>,
       <|"name"->node2,"comm"->com2,"in"->"input2","out"->"output2","p"->{3501,4000}|>,
       <|"name"->node3,"comm"->com3,"in"->"input3","out"->"output3","p"->{4000,4500}|>};
    
  • define calculation server class, where is a sample Mersenne prime number calculation

    new[nam_]:=Module[{ps,pe},
       mersenneQ[n_]:=PrimeQ[2^n-1];
       setv[nam[{s_,e_}]]^:={ps,pe}={s,e};
       calc[nam]^:=Select[Range[ps,pe],mersenneQ]
       ];
    
  • construct instances

    Map[new[#name]&,obj];
    
  • deploy instances to calculation servers

    Map[Save[#in,#name]&,obj];
    Map[Run[#comm]&,obj];
    
  • send message to each instance

    Map[Put[Hold@setv[#name[#p]],#in]&,obj];
    Map[Run[#comm]&,obj];
    
  • start calculation

    Map[Put[Hold@calc[#name],#in]&,obj];
    proc=Map[StartProcess[{$SystemShell,"-c",#comm}]&,obj]
    
  • wait for the process termination (mannualy in this sample code)

    Map[ProcessStatus[#]&,proc]
     {Finished,Finished,Finished}
    
  • gather the results

    Map[FilePrint[#out]&,obj];
     {2203, 2281, 3217}
    {}
    {4253, 4423}
    
POSTED BY: Hirokazu Kobayashi
Answer
16 days ago

Great, I was always wondering how to set this up, thanks for sharing! Could you edit your post and add photos of your setup and Raspberry Pis? It would be very nice to see!

POSTED BY: Marina Shchitova
Answer
16 days ago

If all you need is to be able to use multiple Raspberry Pi computers (or any other computer really) for parallel computation, you can do this using built-in functionality.

If you prefer to use the GUI to set this up, go Preferences -> Parallel -> Remote Kernels.

Click Add Host, and fill out the details. First, fill out the hostname. Then, depending on your setup, you may need to modify the launch command template. You will notice that those numerical slots, like `1` in the launch command template. To find out what they are, evaluate

<< SubKernels`RemoteKernels`

?$RemoteCommand

It is important to use a launch command that will not prompt for a username and password. For this, you will need to set up SSH public key authentication (please google for it).

You may also need to tell Mathematica the username on the remote machine. Load << SubKernels`RemoteKernels` then set $RemoteUser.

Personally, I have not used this GUI method. Instead, I configured remote kernels by typing commands.

I did this by loading << SubKernels`RemoteKernels`, then creating a configuration for each remote computer in the form

x = RemoteMachine[hostname, launchcommand]

The launch command can either be a template like in the GUI configuration (i.e. have slots like `1` in it), or just a plain string (no slots). One you have all these, just append them to $ConfiguredKernels

$ConfiguredKernels = Join[$ConfiguredKernels, {x}]

Once everything is set up, you can simply use LaunchKernels[] and submit parallel evaluations as usual (e.g. ParallelSubmit).

I realize that this is a bit vague, but I do not have a Raspberry Pi at hand to give more detailed instructions. I hope it is still helpful.

POSTED BY: Szabolcs Horvát
Answer
16 days ago

enter image description here - you have earned "Featured Contributor" badge, congratulations !

This is a great post and it has been selected for the curated Staff Picks group. Your profile is now distinguished by a "Featured Contributor" badge and displayed on the "Featured Contributor" board.

POSTED BY: Moderation Team
Answer
12 days ago

Group Abstract Group Abstract