Message Boards Message Boards

0
|
1701 Views
|
2 Replies
|
2 Total Likes
View groups...
Share
Share this post:

ParallelDo and Do results don't match

Posted 1 year ago

Hello, I am asking for a hint.
For example, I have a list of random natural numbers:

t = Table[RandomInteger[{1, 10000}], {10^7}];

I want to count how many prime numbers there are:

howmany = 0;
Do[ If[ PrimeQ[ t[[i]] ], howmany++], {i, 10^7}]

and now the same but using parallel computation:

howmany = 0; 
ParallelDo[ If[ PrimeQ[ t[[i]] ], howmany++], {i, 10^7}]

Unfortunately, the second result is incorrect. My guess is that because there is a problem with accessing vvv at the same time.
Is there any simple solution to this problem? E.g. Each thread has its cold number, and then at the end you can sum them up?

Pozdrawiam
M

POSTED BY: M P
2 Replies

You can try Parallelize[Count[list,pattern]]

POSTED BY: Gianluca Gorni
Posted 1 year ago

According to the documentation for ParallelDo (look under Possible Issues), "Side effects are local to each parallel kernel". The example there is analogous to yours. You can share a variable across kernels with SetSharedVariable.

There is overhead in setting up parallel computations, and I would imagine that sharing a variable adds even more overhead. And so, in your example, performance is actually worse with parallelization than without:

Count[t, _?PrimeQ]
(* Runs in 3 seconds on my machine *)

Do[If[PrimeQ[t[[i]]], howmany++], {i, 10^7}]
(* Runs in 7 seconds *)

ParallelDo[If[PrimeQ[t[[i]]], howmany++], {i, 10^7}]
(* Runs in 9 seconds and produces the wrong answer because howmany isn't shared *)

I don't know if this example is representative of your real scenario, but my suggestions would be:

  • Use the built-in functions designed for list processing first before you try to optimize "by hand".
  • Avoid Do and related constructs unless the scenario actually demands it. Not only are these constructs generally less performant, but they aren't really idiomatic of the Wolfram Language.
  • To provide any benefit, parallel constructs require special care to set up properly, so use them only when the overhead is worth it.
POSTED BY: Eric Rimbey
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract