Message Boards Message Boards

1
|
399 Views
|
10 Replies
|
3 Total Likes
View groups...
Share
Share this post:

Looking for unbuffered Print[]

Posted 26 days ago

Hello Mma users,

I run Mathematica v12.3.1.0 under Windows 10 and I'd like to produce real-time text output.

I have a number of long-running applications and face the following problem: Print[x] only shows at completion or abortion of the running cell. I need to find a way to make the output appear at the time Print[x] is executed. I only need text output, other objects can wait for completion.

Another related question. Some inputs generate a large number of text output, each done via Print[]. Sometimes, the model diverges and start to run astray. At this time, abort (Alt+.) doesn't work and the only way I know to stop the evaluation is to kill the running kernel using Process Explorer or Process Hacker. I know this is rude, and I'd be happy to learn of another, cleaner way to stop a diverging evaluation. Note that if there is no massive Print[] involved, abortion works fine with Alt+.

10 Replies

Try this in a clear session to see what I mean: For[i = 1, i < 10^3, i++, Print[i]]

If you push the limit to, say 10^8 (don't do that!), you're going to wait a lot to see the first Print.

Posted 25 days ago

I'm on a Mac, so maybe this is Windows-specific, but I don't see a delay at all until the first print happens.

But it would help to understand why you're using Print at all. Maybe Monitor would be better?

POSTED BY: Eric Rimbey

I use Print in various places, to display ad-hoc data of very different nature, albeit always in text form.

Monitor is good for displaying subsequent values of the same data from the same place, while the computation makes progress.

BTW, on your Mac, do you get output from Print in the simple For loop posted, even with 10^4 or 10^5 limit? I only get things displayed after completion, even with 10^3.

Posted 25 days ago

Yes. That's what I meant. No matter how big I make the range of the loop, the printing starts immediately.

POSTED BY: Eric Rimbey

By "clear session" do you mean right after starting Mathematica? Then there is a delay due to kernel startup. In older computers it was significant.

POSTED BY: Gianluca Gorni

Yes of course the kernels have to load, but this is quite fast. I mean when everything is ready in a new notebook.

Dear Jean-Christophe,

I think I know what you're talking about. I don't know a way to do what you want, and I don't think it can be done. In fact, in a sense, what you want is being done, at least in theory; however, I think that buffering, or the appearance of buffering, is probably unavoidable. When Print["now"] is executed by the Kernel, it sends a packet ExpressionPacket[BoxData["\"now\"", StandardForm]] to the Front End. The Kernel and Front End run independently. The Kernel does not wait for the Front End to signal it's ready for new input. It keeps on running and may send another packet before the Front End has processed the last one. It takes time for the Front End to insert the cell and re-parsing/re-typesetting the notebook. It's quite possible for the Kernel to generate a great number of packets while the Front End processes just one. There is not a buffer in the Kernel to store these packets to be sent to the Front End at some later time; they are sent immediately.

Of course, all of the unprocessed packets sent by the Kernel are in the input stream of the Front End. And it takes time for the Front End to process them. I don't see any way to avoid such buffering. I think the Front End probably does something for the sake of efficiency when there are a lot of packets. I think it probably collects all of the packets it has received and inserts them before spending time on re-typesetting the notebook and displaying the results. This is probably a good thing, imo, since I will get control of the notebook back faster. It makes Print[] bad for monitoring a process, when that process generates a lot of Print[] output.

Finally, I think abort "works" to stop the Kernel process, once the Front End finds the time to send the abort to the Kernel; however, the Front End may have received thousands of packets that it is still processing.

Gratuituous sharing

Personally, I avoid using Print[], unless I'm sure it will print only a few times. And I curse myself when I'm wrong. :) If I want to monitor a loop, I'll come up with something else. Usually, it's one of two things. If I want to check the data at each iteration, I save the data in some sort of data structure stored in a variable; then I can inspect afterwards. If I'm just monitoring the progress of a loop, then I'll use something like this:

Dynamic[var]
Do[var=i;..., {i, 10^6}]

Of course Dynamic[] is not updated without a delay. It's done with a lower priority than the main Kernel, which is an advantage in that it does not slow down the main process much.

If I want to monitor the progress of a lot of data, say, to see where the program is getting bogged down without waiting for it to quit, then I might dynamically display part of the data. Or display a visualization. Plotting can add a significant amount of time, though. For instance the basic integral below takes almost 5.2 sec. on my laptop. Plotting every update to the data with ListPlot adds about 3 sec. Restricting the updates with UpdateInterval keeps the extra time to a more reasonable value:

data = {};
Dynamic[Refresh[ListPlot@Flatten@data, UpdateInterval -> 0.3, 
  TrackedSymbols :> {}]]
f[x_?NumericQ] := NIntegrate[BesselJ[1, t], {t, 0, x}];
NIntegrate[Abs[Sin[f[x]]], {x, 0, 100}
  , MaxRecursion -> 20
  , Method -> {"GlobalAdaptive", "SymbolicProcessing" -> 0}
  , EvaluationMonitor :> (data = {data, x})] // AbsoluteTiming

(*  < dynamic graphic omitted >  *)
(*  {5.58711, 82.7845}  *)

Straight Graphics[] eliminates the overhead of ListPlot, but you have to do a little more coding. Below showing every update is just a little slower than ListPlot with UpdateInterval -> 0.3 and faster than UpdateInterval -> 0.2:

data = p[]; k = 0;
Dynamic[Graphics[Point[List @@ Flatten@data], Frame -> True, 
  AspectRatio -> 0.6]]
f[x_?NumericQ] := NIntegrate[BesselJ[1, t], {t, 0, x}];
NIntegrate[Abs[Sin[f[x]]], {x, 0, 100}
  , MaxRecursion -> 20 
  , Method -> {"GlobalAdaptive", "SymbolicProcessing" -> 0}
  , EvaluationMonitor :> (data = p[data, p[{++k, x}]])] // AbsoluteTiming

(*  < dynamic graphic omitted >  *)
(*  {5.71038, 82.7845}  *)

The code data = p[data, p[{++k, x}]] creates a structured linked list. Flatten[data] flattens nested p[] to get p[{k1, x1}, {k2, x2},...,{kn, xn}]. It is much more efficient than appending to an array. (Appending copies the data each time a value is appended; the linked list copies the data only when Flatten[data] is executed. Another efficient way is to pre-allocate a large enough array and use a counter variable to keep track of the length; of course, the size of the array needed is often unknown.)

POSTED BY: Michael Rogers

Dear Michael,

Thank you very much for taking the time for a sharp answer.

Yes that's exactly what I suspected: the front-end is submerged by new writes banging at the door. And indeed I don't see an easy issue, lest digging in Mma internals (streams and the like), which is out of question w/o help from dev insiders or very advanced users fluent with such low-level stuff.

I'm going to change my approach and use global variables to monitor externally.

My apps run for days if not weeks, so I want to avoid loosing control of a diverging model going astray.

Kind regards.

Dear Jean-Christophe,

Another option might be to print to a file. You can use Write[] or reset $Output if you want to use Print[]. (See also OpenWrite[].) It must be faster than printing to the Front End. Then you could monitor the file outside of Mathematica.

POSTED BY: Michael Rogers

It seems to me that Print[] prints at the time of its own evaluation. It does not wait for the rest of the cell to be evaluated:

Print["now"]
Pause[2];
Print["then"]
POSTED BY: Gianluca Gorni
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract