Group Abstract Group Abstract

Message Boards Message Boards

Monitor importing process of a CSV file with ProgressIndicator or alike?

Dear all,

I would like to track the importing process of a CSV file. The file is pretty large and it takes hours to load it into Mathematica. To evaluate at which state the uploading process is, I would like to use a Progress Bar or equivalent. Therefore I read at StackOverflow that there is a possibility to track the calculation process of an operation using Monitor or ProcessIndicator. Unfortunately, I have not been able to adapt the code or find the other code which makes this possible. Therefore I am wondering if someone already used this kind of solution to track an importing forces of a CSV file into Mathematica?

I would be very happy if someone could help me, thank you in advance!

Best regards Alex
POSTED BY: Alexander Hempfing
1 Reply
Sort By:

My understanding is that if you are importing that much data, you probably don't want to be using CSV (I would guess that maybe something like HDF would be more appropriate for data at this size) as a file format and you probably don't want to load the entire file into memory but break your computations down and apply them as a stream.

Monitor and ProgressIndicator require that there be some kind of variable to read that indicates how far the evaluation has gone. No such variable exists. You can however import data by using the Read* family of functions and that would give you such a handle.

https://mathematica.stackexchange.com/questions/4640/how-to-monitor-the-progress-of-read
POSTED BY: Sean Clarke
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard