I'm working on an application that which iterates through 6 billion combinations, appending selected items for further use. It is a natural for data parallism. The new DynamicArray datastructure is also a good match since the number of aggregations is quite long and ill-suited for AppendTo.
The DynamicArray structure cannot (currently) be used as Shared Variable so I developed a work-around. I started by adapting the shared variable example in the Help page for ParallelDo to an example using kernel data parallelism:
SetSharedVariable[ppr];
kernels = 4;
ppr = Table[{}, kernels];
LaunchKernels[kernels];
ParallelDo[
Do[
If[PrimeQ[2^i - 1], AppendTo[ppr[[k]], i]], {i, ((k - 1)*1000) + 1,
,13,17,,31,61,89,107,127,521,607,1279,2203,2281,3217}
Within it, I created DynamicArray structures for aggregation, and then stored their Normal contents to the kernel array element:
SetSharedVariable[ppr];
kernels = 4;
ppr = Table[{}, kernels];
LaunchKernels[kernels];
ParallelDo[
tmp = CreateDataStructure["DynamicArray"];
Do[
If[PrimeQ[2^i - 1], tmp["Append", i]], {i, ((k - 1)*1000) + 1,
k*1000}
];
ppr[[k]] = Normal[tmp],
{k, 1, kernels}
];
CloseKernels[];
UnsetShared[ppr];
pr = Flatten[ppr];
Print["pr = ", pr];
pr = {2,3,5,7,13,17,19,31,61,89,107,127,521,607,1279,2203,2281,3217}
Thanks to the Wolfram team for adding this family of data structures to Mathematica.