Message Boards Message Boards

Avoid leak of memory while using FindMinimum?

Posted 8 years ago

Hi everybody, my problem is the following, I need to minimize a certain function "errorfunc" with FindMinimum

(*
errorfunc minimize the L2  HIlbert norm of the difference between TLE \
and TLN*) 
errorfunc[sig_, m_] := Module[{TLN, diff},
   TLN = trT[sig, m] trH[sig, m];
   diff = Total[(TLN - TLE)^2]^(1/2);
   Return[diff];
   Clear[TLN, diff]]; 

FindMinimum needs to evaluate the errofunc thousand times on each couple (sig, m).
The calculation of each function value is lengthy, and requires a reasonable amount of memory. But, for some reason, the memory usage in Mathematica grows steadily until it consumes 83% of my cluster memory (32GB) then the computation continues extremely slowly; why is this? I’ve read something:

https://groups.google.com/forum/#!topic/comp.soft-sys.math.mathematica/UYzbXE615lk

http://stackoverflow.com/questions/6867892/reducing-memory-usage-in-an-extended-mathematica-session

and I thought that the problem was due to the fact that I frequently use Module to define functions. I ‘ve tried to solve the problem setting $HistoryLength=0 and cleaning such variables before exiting Module, but nothing seems to change, the leakage still remains. Has anybody an idea how to avoid this leakage? Attached you can find the entire code. (I'm usign Mathematica v10.0)

Thanks in advance for any suggestions, Luisa

Attachments:
POSTED BY: Luisa TESTA
2 Replies
Posted 8 years ago

I ‘ve tried to solve the problem setting $HistoryLength=0

As I understand, you tried to set

$HistoryLength = 0

in the master kernel only. But the slave kernels still have infinity history length what you can check by evaluating

 ParallelEvaluate[$HistoryLength]

I recommend you setting zero history length in the slave kernels too:

ParallelEvaluate[$HistoryLength = 0]
POSTED BY: Alexey Popkov
Posted 8 years ago

Module does cause memory leaks in a number of specific cases, but I don't think the memory leak you describe is due to Module. I recommend you reading this (specifically the penultimate paragraph from the second post by Leonid Shifrin and posts linked therein) for additional information about situations when Module can cause memory leaks. Also you can find useful the following threads:

POSTED BY: Alexey Popkov
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract