Message Boards Message Boards

0
|
5464 Views
|
5 Replies
|
3 Total Likes
View groups...
Share
Share this post:

Why Version 10 seems to be much more memory hungry?

Nasser M. Abbasi, student
Posted 11 years ago

I had a simple matrix performance rank test that ran fine in version 9.01 on windows 7 up to n=8000. When I run the same test on version 10, when n=7500 Mathematica now started consuming all memory on the PC (I have 16 GB of ram) and actually the whole PC hanged for long time. I can hear the disk thrashing and the whole process was very bad. After 30 minutes or so, I was about to reboot the PC. But was finally able to access the task manager and terminate Mathematica process.

The test is simple, here it is

Remove["Global`*"];
Share[];
n = 7500; (*values above hanged my PC*)
m = Table[Random[], {i, 1, n}, {j, 1, n}];
AbsoluteTiming[MatrixRank[m];]

I also noticed this memory problem on other tests. I have a large differential equation test case where Mathematica hanged on many cases. All memory issues. I do not know what changed. As I said, I run the same test, on the same exact PC with version 9. To see more details of the test, here is a link

May be someone can investigate this more or have an idea what is the problem. Does this problem also show up on the Mac? What about Linux?

Warning: Do not run the above on your PC without first saving all you work, all you documents from other application. Mathematica can and will hang the PC unless you have zillion GB of memory. I am going to buy more memory myself and add another 16 GB and try this test again.

But for me, this is a regression in Mathematica performance from version 9.01. A small follow up question: Is there a way to tell Mathematica, as a global options or setting, not to use system memory more than some limit? For example, tell it to terminate any computation if Mathematica process (FF or Kernel) has allocated more than say 6GB or memory). If there is such a way, then I can set this limit in my preferences and not have to worry about the possibility of running out of PC memory again when using Mathematica. I assume Mathematica internally is able to determine how much system RAM is consumed at any one point.
POSTED BY: Nasser M. Abbasi
5 Replies
Sort By:

One can wrap the desired operations within a variety of *Constrained operations, where * might be Memory, Time, Pixel etc (on Mathematica 7).

MemoryConstrained[ some operation ]
POSTED BY: Isaac Abraham

Thanks, I know about these functions, but I was looking at something at the process level, so do not have to specific this each time for each command. Basically tell Mathematica Kernel itself, not to use more memory than some specific limit since in some large computation, one does not always know which operation will consume how much memory. I think this is a OS problem as well. I am really surprised that Windows allows any one process to just take as much memory as it wants with no restriction. But this is windows. I do not know how this behaves on Linux or the Mac.
POSTED BY: Nasser M. Abbasi

Starting the code with 

$HistoryLength=0

is a better way to keep unsaved intermediate values from cluttering memory than with 

Remove["Global`*"];

RandomReal has been around since 6.0. (Faster and more memory-efficient than Table[Random[],...) 

In[1]:= n = 7000;                                                                       

In[2]:= $HistoryLength=0;                                                               

In[3]:= Timing[ Table[Random[], {i, 1, n}, {j, 1, n}]; ]                                
Out[3]= {17.960200, Null}

In[4]:= Timing[ RandomReal[{}, {n, n}]; ]                                               
Out[4]= {1.164463, Null}

In[5]:= MaxMemoryUsed[]                                                                 
Out[5]= 1204929176

In[6]:= m=RandomReal[{}, {n, n}];                                                       

In[7]:= AbsoluteTiming[MatrixRank[m];]                                                  

Out[7]= {431.394567, Null}

In[8]:= MaxMemoryUsed[]                                                                 

Out[8]= 1204929176
POSTED BY: Bruce Miller

$HistoryLength=0 and RandomReal[{} seems to have made big difference on memory. I re-run the test and got simliar CPU values as with v 9.01, but without the memory problem. I did not want to change the test since I used the same test before 2 times to compare with. But since this change only affected memory and not CPU which is what I am testing for, I can do that and still be fair to the other two programs I am comparing with.

Now Mathematica is again able to do n=7500 and n=8000 as it did in V9. Updated the test results.

Thanks.
POSTED BY: Nasser M. Abbasi

I do not know what is behind this particular issue. With regard to Linux handling of memory issues, on Ubuntu at least I can say that it is miserable (at least with default settings). Actually it has been miserable on every Linux I have used, so it's not specific to Ubuntu. I often wish there were a dedicated part of reptilian brain in the OS that was reserved for killing rogue processes, and moreover was always reachable (without waiting 30 minutes and getting a lucky process swap). Probably this is an engineering impossibility.

(Had to reboot twice last week, and still annoyed.)
POSTED BY: Daniel Lichtblau
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract