How to Parallelize a Calculation

Questions
, ,
1

Hello everyone,

I’m beginner in Mathematica and I am trying to implement my first parallelized calculation.

I’ve a function of 7 variables and I want to minimize it in a specific range for the variables.

So I define 7 ‘for loop’ nested in each other to calculate the value of the function by making discrete steps for each variable

For exemple for to loops I would have something like

For [ counter1, counter1<max1,counter1++;
    For[counter2, counter2<max2,counter2++;
        f(x1+counter1*x1step,x2+counter2*x2step);
    ]
]

I would like to use my CPUs to parallelize the calculation abd to limit the number of the loop required. I would like to fix the value of the first parameters and perform the calculation only in the second loop by using different CPU and then compare the results given by the different CPUs.

For example I would like to do :

CPU1
x1=1
 For[counter2, counter2<max2,counter2++;
        f(x1,x2+counter2*x2step);]


CPU2
x1=2
 For[counter2, counter2<max2,counter2++;
        f(x1,x2+counter2*x2step);]


CPU3
x1=3
 For[counter2, counter2<max2,counter2++;
        f(x1,x2+counter2*x2step);]

And then compare the results (even not automatically)

I know that a FIndMinimun function exist and that my way of minimizing the function is cleary not efficient but for personal reason I want and need to use this program’s structure with many ‘for loop’

Could you help me to do that step by step because I haven’t understood the Wolfram’s tutorial ?

Thank you very much for your help.

Corentin

2

Parallel programming is sometimes very difficult. In the cases where it is fairly straightforward, the Wolfram Language makes it very easy by providing functions like ParallelMap, ParallelTable, etc. These simple cases of parallelization can be seen in the documentation for “DataParallelism”. Data Parallelism—Wolfram Documentation

Please first make sure you understand data parallelism.

Writing parallel code that isn’t data parallel is inherently difficult. Most people study this as an upper level computer science course in college. This is not a project for a beginning programmer.

Have you written a non-parallel verison of your objective function (the function to be minimized?) yet? That would be the first step.

3

Yes I already have implemented a non parralel-version of my program so I build my objective function but I don’t really understand how parallel calculation work with Mathematica even reading the help (may be I simply don’t understand it at all). If it possible I need your help to have an example

I’m pretty sure that making parallel calculation with my program is not difficult

4

  1. The function you posted doesn’t seem quite right. See output to:

    f[x_, y_] := {Sin, Cos[y + 1]}; With[{max1 = 10, max2 = 4, x1 = 0,
    x2 = 2, x1step = 1, x2step = 1}, Module[{counter1 = 1, counter2 = 1},
    For[counter1, counter1 < max1, counter1++,
    For[counter2, counter2 < max2, counter2++,
    Print@f[x1 + counter1x1step, x2 + counter2x2step]]]
    ]]
    =>
    {Sin[1],Cos[4]}

    {Sin[1],Cos[5]}

    {Sin[1],Cos[6]}

  2. Not sure that For statements nest all that well. Try it for a simple case.

  3. Parallel computing sounds good, but – it takes time to transfer information to each core, and time to retrieve it and assemble it into the result. So – unless the function you want to evaluate takes much longer than the setup and knockdown process, you actually lose time by computing in parallel. additionally, consider that the non-parallel versions of of Table and Array have been considerably optimized.

So, for this easily evaluated function:
Serial version takes 2.9 seconds:

ParallelArray[{Sin@(1.*#1), Cos@(1. #2)} &, {3*^3, 4*^3}, {1, 10}];

but the serial version takes 4.0. seconds.

ParallelArray[{Sin@(1.*#1), Cos@(1. #2)} &, {3*^3, 4*^3}, {1, 10}

I would suggest:

a) Try to formulate and execute the problem in serial first, for a small case. Do not use the nested For. Try for Array[], as shown above. Note: ParallelTable won’t accept the above evaluation, but Array will.

b) Then try the Parallel version of Array and see if it helps. NOTE: Your parallel computations must be independent of each other for the above method to work. If they modify a common data structure, things get more complicated.

5

In what language/framework are you familiar with parallel programming? Mathematica is basically the easiest programming language to do data parallelism in.

Do you understand how Mathematica does data parallelism? This is the first step.

As usual, the first step is to create a very simple version of your problem. Please create a simple example of problem similar to yours. If you show us the code for a simple example of a sequential program, this will help us explain how to parallelize it. I should warn you that Parallel function optimization is rarely easy.

6

I never performed parallel computation before…

But I find an other way to achieve my purpose using FindMinimum, even if it’s extremely sensitive to starting points.

Thanks for our help anyway

7

FindMinimum is supposed to be sensitive to initial starting values. It is a local optimizer. So for certain functions it is supposed to be very sensitive. See the wikipedia page on Newton Fractals for a classic example (Newton fractal - Wikipedia).

You probably want global optimization, which is NMinimize. This attempts to minimize the function globally without being sensitive to inital values.