ParallelTable vs ParallelSubmit

Questions
, , ,
1

I want to solve two ODEs in parallel which almost need the same time to solve. So I expected the parallel program to have approximately 50% of the running time of the serial one. This is true when I use ParallelSubmit but when I use ParallelTable the speedup is much lower.

Method          Walltime
Serial          3m10.647
ParallelSubmit  1m42.175s
ParallelTable   2m48.830s

Here are the codes I use:

ParallelSubmit

f := (
        d := 0.0000001;
        {time,tbl} = AbsoluteTiming[Table[{step, Timing[NDSolve[{x'[t] == x[t]^2 - x[t]^3, x[0] == d}, x, {t, 0, 2/d}, MaxStepSize->step, MaxSteps->10000000]]}, {step,5,5,-0.4}]];
)

g := (
        d := 0.0000001;
        {time,tbl} = AbsoluteTiming[Table[{step, Timing[NDSolve[{x'[t] == x[t]^2 - x[t]^3, x[0] == d}, x, {t, 0, 2/d}, MaxStepSize->step, MaxSteps->10000000]]}, {step,4.6,4.6,-0.4}]];
)

LaunchKernels[2];
DistributeDefinitions[f, g];
e = {ParallelSubmit[f], ParallelSubmit[g]};
WaitAll[e];

ParallelTable

d := 0.0000001;

LaunchKernels[2];
{time,tbl} = AbsoluteTiming[ParallelTable[{step, Timing[NDSolve[{x'[t] == x[t]^2 - x[t]^3, x[0] == d}, x, {t, 0, 2/d}, MaxStepSize->step, MaxSteps->10000000]]}, {step,5,4.6,-0.4}]];
CloseKernels[];

These diagrams show the CPU usage of both Methods:

ParallelSubmit

parallelsubmit.png
parallelsubmit.png1188Ă—177 29.9 KB

ParallelTable

paralleltable.png
paralleltable.png1188Ă—180 91.8 KB

ParallelSubmit looks as expected but ParallelTable looks like it uses different cores at different times. Especially at the end the picture is quite chaotic.

Is there a better way to use ParallelTable? Maybe an option I am missing?

2

In your ParallelTable version, you didn’t distribute the value of “d”. When I do that (and get rid of extraneous stuff in your code), the timing results for the 2 cases are nearly identical.

3

I think per ParallelTable’s doc page it should distribute the definition automatically because DistributedContexts -> Automatic is preset. Omitting d should not affect the performance.

4

You are right there were some unnecessary lines in my codes. I deleted them but that didn’t make any difference in the runtimes. I tried to distribute the definition of d but that made no difference either. I tested it with Mathematica 9.0 and 10.0.1. Which version did you use?

5

I looked at this more last week. I suspect that NDSolve rather than parallel tools is the source of the problem (your ode results in something quite close to a step function. Quite challenging to solve!). I tried several different implementations using ParallelSubmit, ParallelEvaluate, and ParallelTable. Computation time sometimes showed speedup and sometimes not. I also tried v9 and v10 and saw no significant differences between the two.

I will ask some people more familiar with NDSolve to take a look.