Message Boards Message Boards

0
|
5969 Views
|
3 Replies
|
1 Total Likes
View groups...
Share
Share this post:

Sow and reap in parallel submit with outside variable use

Posted 10 years ago

Hello,

I try to use reap and sow to accumulated values during a calculation. It is recommended because it is said to be faster than AppenTo. However, I would like to utilize all my cpu cores by using parallelsubmit on various reap expressions with nested for and sow. It seems that this does not work properly. Please take a look at the following example:

end = 5
Reap[
 For[t = 0, t < end, t++,
  Sow[t]
  ]
 ]
WaitAll[%]

returns: {Null, {{0, 1, 2, 3, 4}}} -> Good

end = 5
ParallelSubmit[
 Reap[
  For[t = 0, t < end, t++,
   Sow[t]
   ]
  ]
 ]
WaitAll[%]

returns: {Null, {}} -> Bad

However, if I replace the end variable with its value like so:

ParallelSubmit[
 Reap[
  For[t = 0, t < 5, t++,
   Sow[t]
   ]
  ]
 ]
WaitAll[%]

returns: {Null, {{0, 1, 2, 3, 4}}} -> Good

Why is that?

POSTED BY: Mathias Breuss
3 Replies

Probably, the time required to do all the parallel staff (kernels communication and other) is greater then plain sequential computation time. The benefit from parallel computations can be seen, for example, for ParallelTable[] when it's arg takes some decent time to be evaluated for a single iteration:

ParallelTable[Pause[1]; 0.,{4.}]; // AbsoluteTiming
Table[Pause[1];0.,{4.}]; // AbsoluteTiming
(*{1.009058, Null}*)
(*{4.003229, Null}*)

I.M.

POSTED BY: Ivan Morozov
Posted 10 years ago

Hi Ivan,

thanks for your answer. That solved the problem indeed.

However, lets say I put end = 2000, using ParallelSubmit it takes 2 seconds to compute the result. While it barely takes any time when I run

Reap[For[t = 0, t < end, t++, Sow[t]]]

without using ParallelSubmit.

Any Idea why that might be slower?

POSTED BY: Mathias Breuss

Hi,

'end' var should be shared:

SetSharedVariable[end] ;
end = 5 ;
ParallelSubmit[Reap[For[t = 0, t < end, t++, Sow[t]]]] ;
WaitAll[%]

I.M.

POSTED BY: Ivan Morozov
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract