Group Abstract Group Abstract

Message Boards Message Boards

0
|
9.3K Views
|
6 Replies
|
2 Total Likes
View groups...
Share
Share this post:

Speed problem: Import and post-process 1000 images

Peter Parker, Research
Posted 9 years ago
Attachments:
POSTED BY: Peter Parker
6 Replies
Sort By:

Hahaha, true :-) It is not very fast. I am sure it can be done much faster. But how much effort you have to put into this just to be a little bit faster....? Is it worth it? But I agree, would be interesting to see and I am very curious about it.

POSTED BY: Peter Parker

Actually, a speedup factor of 2 isn't very great. I would love to see a faster solution.
POSTED BY: Szabolcs Horvát

Hi Szabolcs

Indeed, this technique is definitely faster. Thanks a lot. I think with this improvement I can now say, the speed is reasonably okay. Yesterday it was quite slow but with your code and "Quit Kernel" the speed is much higher.

Thanks a lot :-)

Peter

POSTED BY: Peter Parker

I was unable to get ReadList to handle commas as separators, but the following at least provides a ~2.3x speed boost to importing on my computer.

csvRead[file_] :=
 Module[{stream, data},
  stream = StringToStream@StringReplace[ReadString[file], "," -> "\t"];
  data = ReadList[stream, Real, RecordLists -> True];
  Close[stream];
  data
  ]

In[59]:= csvRead["~/Downloads/11.4449.csv"]; // AbsoluteTiming

Out[59]= {0.664697, Null}

In[60]:= Import["~/Downloads/11.4449.csv", "CSV"]; // AbsoluteTiming

Out[60]= {1.57064, Null}
POSTED BY: Szabolcs Horvát
POSTED BY: Peter Parker
POSTED BY: Szabolcs Horvát
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard