Message Boards Message Boards

1
|
6406 Views
|
2 Replies
|
3 Total Likes
View groups...
Share
Share this post:

Compress data that has been saved using Put?

Posted 5 years ago

Just want to confirm that the Put function does not have an option for compressing large expressions (associations, upwards of 1GB). So, if I compressed the saved files after the fact, will Get automatically decompress them? There is nothing to that effect in the documentation, so I am assuming not.

Greg

POSTED BY: Gregory Lypny
2 Replies
Posted 5 years ago

Hi Aaron,

This rocks! Thank you. Was able to cut the size of the saved (test) files by about 80 per cent. I'm going with your option 2. Do you happen to know what the format of the compressed file is? The reason I ask is so I can add an extension that allows me to distinguish between my compressed and uncompressed data files. If not, I can just add a little something to the file base names.

Thanks once again,

Greg

POSTED BY: Gregory Lypny

I think you have 2 options:

1) Use Import instead of Get:

In[131]:= ExampleData[{"Text", "AliceInWonderland"}] >> "~/alice.m"
(* From the command line:
$ gzip ~/alice.m *)

In[136]:= Import@"~/alice.m.gz" // StringTake[#, 100] &

Out[136]= "I--DOWN THE RABBIT-HOLE Alice was beginning to get very tired of sitting by \
her sister on the bank, "

2) Comprress the data first, then use Put:

    In[137]:= Compress[ExampleData[{"Text", "AliceInWonderland"}]] >> "~/alice-compressed.m"

    In[139]:= Uncompress[Get@"~/alice-compressed.m"] // StringTake[#, 100] &

    Out[139]= "I--DOWN THE RABBIT-HOLE Alice was beginning to get very tired of sitting by \
    her sister on the bank, "
POSTED BY: Aaron Enright
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract