Message Boards Message Boards

0
|
480 Views
|
4 Replies
|
4 Total Likes
View groups...
Share
Share this post:

Too much memory usage when working with lists, but not when working with datasets

Ehud Behar
Posted 2 months ago

I’m encountering a problem in Mathematica while working with two large lists, list1 and list2, each containing 3600×2 cells of numerical values. After performing some arithmetic operations between these lists, both symbols became undefined, requiring me to redefine them and evaluate the previous cells again.

I noticed that when I create a Dataset object from those lists, the issue does not happen.

I suspect this might be related to memory consumption on my computer.

Has anyone else experienced something similar?

(I am on Windows 10, 32GB RAM with 11th Gen Intel i7-11700 @ 2.50GHz)
POSTED BY: Ehud Behar
4 Replies
Sort By:

If it happens in ListPlot that sounds like a crash bug. Maybe send example to Tech Support.
POSTED BY: Daniel Lichtblau

3600×2 of machine real numbers takes up <60 KB packed or <350 KB unpacked. It's a small amount of memory. It's doesn't rule out a memory-related problem, since who knows what sort of operations you're performing. But even DistanceMatrix[] on the array takes up only 100 MB (packed).

I've experienced silent kernel crashes. I assume that's what you have. I'm not sure I've ever know what causes them. Mathematica can catch some things that request too much memory. I've sometimes guessed that an error occurs in a library, and the kernel is not prepared to deal with the consequences.
POSTED BY: Michael Rogers
Ehud Behar
Ehud Behar
Posted 2 months ago

Sorry, I forgot to mention it in the original post: the symbols are reset also when doing ListPlot[{list1,list2}]. Does that take any considerable memory?
POSTED BY: Ehud Behar

You might intersperse code with MaxMemoryUsed[] to see if memory consumption is the actual issue.
POSTED BY: Daniel Lichtblau
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract