Group Abstract Group Abstract

Message Boards Message Boards

3
|
8.6K Views
|
5 Replies
|
9 Total Likes
View groups...
Share
Share this post:

How I saved Gigabytes of memory or the Discovery of FAT and thin lists

Posted 11 years ago
POSTED BY: Douglas Kubler
5 Replies

Yes, Map uses autocompilation for lists over a certain size and they will be packed if possible. The direct way would be

packed = Developer`ToPackedArray[unpacked];
POSTED BY: Ilian Gachevski
Posted 11 years ago

And to be different Map decides the result should be packed. I'll be using this.

packed = Map[Identity, unpacked];
POSTED BY: Douglas Kubler

myList is not a packed array and I guessFlatten therefore decides the result should be unpacked. Had the thing been packed to begin with there would be no unpacking by Flatten. Try it e.g. with

myList2 = RandomReal[1, {1000000, 10}];

As for Join, it is in fact seeing packed arrays since the component sublists of myList are packed. Ergo, a packed result.

POSTED BY: Daniel Lichtblau

Very interesting! I'm interested to know the cause of this. I sometimes also work with very big files...

POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard