Dear Jean-Christophe,
I think I know what you're talking about. I don't know a way to do what you want, and I don't think it can be done. In fact, in a sense, what you want is being done, at least in theory; however, I think that buffering, or the appearance of buffering, is probably unavoidable. When Print["now"]
is executed by the Kernel, it sends a packet ExpressionPacket[BoxData["\"now\"", StandardForm]]
to the Front End. The Kernel and Front End run independently. The Kernel does not wait for the Front End to signal it's ready for new input. It keeps on running and may send another packet before the Front End has processed the last one. It takes time for the Front End to insert the cell and re-parsing/re-typesetting the notebook. It's quite possible for the Kernel to generate a great number of packets while the Front End processes just one. There is not a buffer in the Kernel to store these packets to be sent to the Front End at some later time; they are sent immediately.
Of course, all of the unprocessed packets sent by the Kernel are in the input stream of the Front End. And it takes time for the Front End to process them. I don't see any way to avoid such buffering. I think the Front End probably does something for the sake of efficiency when there are a lot of packets. I think it probably collects all of the packets it has received and inserts them before spending time on re-typesetting the notebook and displaying the results. This is probably a good thing, imo, since I will get control of the notebook back faster. It makes Print[]
bad for monitoring a process, when that process generates a lot of Print[]
output.
Finally, I think abort "works" to stop the Kernel process, once the Front End finds the time to send the abort to the Kernel; however, the Front End may have received thousands of packets that it is still processing.
Gratuituous sharing
Personally, I avoid using Print[]
, unless I'm sure it will print only a few times. And I curse myself when I'm wrong. :) If I want to monitor a loop, I'll come up with something else. Usually, it's one of two things. If I want to check the data at each iteration, I save the data in some sort of data structure stored in a variable; then I can inspect afterwards. If I'm just monitoring the progress of a loop, then I'll use something like this:
Dynamic[var]
Do[var=i;..., {i, 10^6}]
Of course Dynamic[]
is not updated without a delay. It's done with a lower priority than the main Kernel, which is an advantage in that it does not slow down the main process much.
If I want to monitor the progress of a lot of data, say, to see where the program is getting bogged down without waiting for it to quit, then I might dynamically display part of the data. Or display a visualization. Plotting can add a significant amount of time, though. For instance the basic integral below takes almost 5.2 sec. on my laptop. Plotting every update to the data with ListPlot
adds about 3 sec. Restricting the updates with UpdateInterval
keeps the extra time to a more reasonable value:
data = {};
Dynamic[Refresh[ListPlot@Flatten@data, UpdateInterval -> 0.3,
TrackedSymbols :> {}]]
f[x_?NumericQ] := NIntegrate[BesselJ[1, t], {t, 0, x}];
NIntegrate[Abs[Sin[f[x]]], {x, 0, 100}
, MaxRecursion -> 20
, Method -> {"GlobalAdaptive", "SymbolicProcessing" -> 0}
, EvaluationMonitor :> (data = {data, x})] // AbsoluteTiming
(* < dynamic graphic omitted > *)
(* {5.58711, 82.7845} *)
Straight Graphics[]
eliminates the overhead of ListPlot
, but you have to do a little more coding. Below showing every update is just a little slower than ListPlot
with UpdateInterval -> 0.3
and faster than UpdateInterval -> 0.2
:
data = p[]; k = 0;
Dynamic[Graphics[Point[List @@ Flatten@data], Frame -> True,
AspectRatio -> 0.6]]
f[x_?NumericQ] := NIntegrate[BesselJ[1, t], {t, 0, x}];
NIntegrate[Abs[Sin[f[x]]], {x, 0, 100}
, MaxRecursion -> 20
, Method -> {"GlobalAdaptive", "SymbolicProcessing" -> 0}
, EvaluationMonitor :> (data = p[data, p[{++k, x}]])] // AbsoluteTiming
(* < dynamic graphic omitted > *)
(* {5.71038, 82.7845} *)
The code data = p[data, p[{++k, x}]]
creates a structured linked list. Flatten[data]
flattens nested p[]
to get p[{k1, x1}, {k2, x2},...,{kn, xn}]
. It is much more efficient than appending to an array. (Appending copies the data each time a value is appended; the linked list copies the data only when Flatten[data]
is executed. Another efficient way is to pre-allocate a large enough array and use a counter variable to keep track of the length; of course, the size of the array needed is often unknown.)