[?] How it works ParallelTable[Print...]?

Questions
1

Dear Community,
I have a very simple question relating to ParallelTable[] function. The following code works as it must.

In[1]:= ParallelTable[k, {k, 10}]
Out[1]= {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

But, if I use Print[], it works as it follows:

In[2]:= ParallelTable[Print[k], {k, 10}];
(kernel 2) 1
(kernel 1) 4
(kernel 2) 2
(kernel 1) 5
(kernel 2) 3
(kernel 1) 6
(kernel 2) 7
(kernel 1) 9
(kernel 2) 8
(kernel 1) 10

Can someone explain how it works?

One additional code, the result of which looks very strange:

In[3]:= i = 0; ParallelTable[Print["k = ", k, " \[Rule] i = ", ++i], {k, 10}];
 (kernel 2) k = 1 -> i = 26
 (kernel 1) k = 4 -> i = 26
 (kernel 2) k = 2 -> i = 27
 (kernel 1) k = 5 -> i = 27
 (kernel 2) k = 3 -> i = 28
 (kernel 1) k = 6 -> i = 28
 (kernel 2) k = 7 -> i = 29
 (kernel 1) k = 9 -> i = 29
 (kernel 2) k = 8 -> i = 30
 (kernel 1) k = 10 -> i = 30

Why the values of variable i are so strange?

2

Can someone explain how it works?

It works in parallel, therefore there are no guarantees about the order of evaluation.

Why the values of variable i are so strange?

Because each subkernel (paralell thread) has its own private copy of i. The value 0 is distributed to them only once.

These are basic problems that come up in all forms of parallel programming. As you learn more about the topic, you will find countless discussions (even books) on them. At this point it may be a good idea to read through the Parallel Computing Tools User Guide, though I do not remember to what extent it addresses these problems.

3

ParallelTable treats the iterator k as a shared variable. While the individual expressions may evaluate on any kernel, the controlling kernel assembles the individual Output results into the expected ordering.

Print produces a “Print” cell instead of Output. The subkernels (compute kernels) do not have a notebook interface to produce cells. The parallel functionality has been designed to send the Print output back to the controlling kernel, labeled according to the subkernel that evaluated the expression.

The mixed ordering of the print cells is known as a race condition. Because the print cells are not Output, the controlling kernel does not re-order them.

When you set i=0 in the controlling kernel, that definition is only sent to all the subkernels if i is not yet defined on the subkernels! Your subkernels must have different definitions for the variable i (based on your output they started with i=25). ParallelTable has the attribute HoldAll. Thus, your variable i in the body of ParallelTable is only known by the individual subkernels and updates independently.

4

Thank you @Szabolcs and @Kevin for your valuable answers!