Message Boards Message Boards

0
|
4328 Views
|
1 Reply
|
2 Total Likes
View groups...
Share
Share this post:

Using many big numbers and export them as an array to external file,?

Posted 8 years ago

Hello, Community!

Main problem - i have a big array (10^6) of big different numbers (10^30-10^70), countered before. All array is nesessary for computation on the next step (parallel counting is impossible, only step after step).

The question: how to use all array - it's too big.

I try to export array to external file, and then use it on the next step. Export is OK. Then use Import (recieve many redundant words), Get (many redundant words, some of them differs from the first variant),FilePrint (not many redundant words).

I need my numbers, but recieve words. Try to change Size Limit (after words "A very large output was generated. Here is a sample of it:") - no effect.

What is wrong? If exists another way to use big array in computation - please inform me.

Attachments:
Posted 8 years ago

Perhaps Mathematica cannot use one million 70 digit numbers. Test this. I start Mathematica fresh and do these steps:

In[1]:= MemoryInUse[]

Out[1]= 25728400

In[2]:= z = RandomInteger[{10^30, 10^70}, 10^6];

In[3]:= MemoryInUse[]

Out[3]= 108026264

In[4]:= 108026264 - 25728400

Out[4]= 82297864

So one million 70 digit numbers have use 82297864 bytes and on today's computers that is a fairly small number. Notice that I carefully did not display those million numbers. To do that might require more memory.

Can I do calculations on that million numbers without needing far far more memory? What calculations? I can only guess.

In[5]:= For[i = 1, i < 10^6 - 10, i += 10,
 z[[i]] = z[[i]]+2*z[[i+1]]-z[[i+2]]+z[[i+5]]*z[[i+4]]-z[[i+7]]*z[[i+6]]-3*z[[i+8]]*z[[i+9]]+z[[i+10]]
 ]

In[6]:= MemoryInUse[]

Out[6]= 125644504

In[7]:= 125644504 - 82297864

Out[7]= 43346640

Again a small amount more memory is used.

I can export that

Export["z.txt", z]

It takes time, but only a small amount more memory to export 75 million bytes of text.

No one can guess what calculations you are doing or why you are running out of memory because you have not shown that.

Try doing the calculations, never display the million 70 digit numbers on the screen and export the result.

If that works then perhaps you are done. If not then make another post describing simply and clearly everything you are doing and perhaps someone can look at your code and see how to make it take less memory.

Good general rule for Mathematica users: Always buy WAY too much memory. Memory is inexpensive. Buy 16 gigabytes or even 32 gigabytes. If you know enough or you spend enough time trying to minimize use of memory then perhaps you can need less. If you have more you do not need to think about memory limits as much. It is unfortunate that there are not inexpensive current motherboards which will allow you to use 128 gigabytes.

POSTED BY: Bill Simpson
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract