# Estimated remaining time: monitoring lengthy computations

GROUPS:
 Vitaliy Kaurov 8 Votes I sometimes run lengthy computations, overnight or even over a few days. We have built in Monitor function to track the progress of a variable. But I often find it useful to estimate how much time is left till a computation is done. It is easy to estimate, but it’d be nice to have it monitored automatically. Like many OS do, for example, for files transfer. Below I give a prototype of a function that does it. The idea is very simple and assumes that computations are approximately uniform in time. A typical usage case would be some image processing of large directory of images. We compute the rate based on variable (e.g. number of images) progress and time passed. The rate allows estimating remaining time. Here is the legend for its arguments:content – the computation to trackticker – the variable to trackmin – minimum (starting) value of tickermax – maximum (ending) value of tickerperiod – periodicity to check the status of computation with“period” is needed because monitoring in principle slows down the original computation. To reduce the slow-down check for the computation status with longer (than single iteration) time intervals. Clear["Global*"]; SetAttributes[RemainingTime, HoldAll]; RemainingTime[content_, ticker_, min_, max_, period_] :=  Block[{T0 = AbsoluteTime[], toTimE, timE},   ticker = min;   Monitor[content,    Refresh[     timE = AbsoluteTime[] - T0;     toTimE = (max - min) timE/(ticker - min + (max - min) 10^-6);    Panel@Grid[      {{"current value  ", ProgressIndicator[ticker, {min, max}],         ticker},       {"remaining time ",         ProgressIndicator[toTimE - timE, {0, toTimE}],         Round[toTimE - timE]}},      Alignment -> Left],    UpdateInterval -> period, TrackedSymbols -> {}]   ]  ]Now let see how the function works. For uniform in time computations it works fine:RemainingTime[Do[Pause[0.01], {i, 1, 500}], i, 1, 500, .1]For non-uniform in time computations, like finding prime numbers, we can obviously see some trouble: RemainingTime[ primes = {}; Do[If[PrimeQ[2^i - 1], AppendTo[primes, i]], {i, 1000, 5000}]; primes , i, 1000, 5000, .1]Let me know any suggestions or post any improvements or variations. For instance, how do we account for non-uniform in time computations?
4 years ago
5 Replies
 Patrick Stevens 1 Vote You could perhaps do a FindFit on the times each successive result took, and extrapolate using that?
4 years ago
 @PatrickStevens Yes, good idea, we could do that type of forecasting, but that would require accumulation of points and time spending on fitting. This will slow down the monitored computation. So I guess the question is how do we find a proper balance between small enough slow down and sophisticated enough forecasting. I wonder if there are some standard algorithms for this.