Group Abstract Group Abstract

Message Boards Message Boards

0
|
2.8K Views
|
2 Replies
|
2 Total Likes
View groups...
Share
Share this post:
GROUPS:

Using DynamicArray in Parallel Kernels

Posted 1 year ago

I'm working on an application that which iterates through 6 billion combinations, appending selected items for further use. It is a natural for data parallism. The new DynamicArray datastructure is also a good match since the number of aggregations is quite long and ill-suited for AppendTo.

The DynamicArray structure cannot (currently) be used as Shared Variable so I developed a work-around. I started by adapting the shared variable example in the Help page for ParallelDo to an example using kernel data parallelism:

SetSharedVariable[ppr];
kernels = 4;
ppr = Table[{}, kernels];
LaunchKernels[kernels];
ParallelDo[
  Do[
   If[PrimeQ[2^i - 1], AppendTo[ppr[[k]], i]], {i, ((k - 1)*1000) + 1,
     ,13,17,,31,61,89,107,127,521,607,1279,2203,2281,3217}

Within it, I created DynamicArray structures for aggregation, and then stored their Normal contents to the kernel array element:

SetSharedVariable[ppr];
kernels = 4;
ppr = Table[{}, kernels];
LaunchKernels[kernels];
ParallelDo[
  tmp = CreateDataStructure["DynamicArray"];
  Do[
   If[PrimeQ[2^i - 1], tmp["Append", i]], {i, ((k - 1)*1000) + 1, 
    k*1000}
   ];
  ppr[[k]] = Normal[tmp],
  {k, 1, kernels}
  ];
CloseKernels[];
UnsetShared[ppr];
pr = Flatten[ppr];
Print["pr = ", pr];

pr = {2,3,5,7,13,17,19,31,61,89,107,127,521,607,1279,2203,2281,3217}

Thanks to the Wolfram team for adding this family of data structures to Mathematica.

POSTED BY: Richard Frost
2 Replies

It’s a good structure to have. It was implicitly there already, in Sow/Reap. This new structure makes it more obvious and is better suited to a procedural style of coding. Much easier than trying to remember the layout of Reap results.

POSTED BY: Daniel Lichtblau
POSTED BY: Richard Frost
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard