Message Boards

WOLFRAM COMMUNITY

4812 Views

4 Replies

0 Total Likes

View groups...

Follow this post

Share this post:

GROUPS:

ListContourPlot from a list with about 7000000 triplets (x,y,z)?

Jan Baetens

Jan Baetens, Ghent University

Posted 8 years ago

I wanna create a ListContourPlot from a list with about 7000000 triplets (x,y,z) using the following piece of code: a = ToExpression[Import["TIBBuil4totaal.txt", "List"]]; b = Map[Join[#[[1]], #[[2]]] &, Partition[ Riffle[Partition[Riffle[a[[All, 1]], a[[All, 2]]], 2], Partition[a[[All, 3]], 1]], 2]]; t = {0.000051606234793646216`, 0.000013374424284682572`, 7.140323683145797`^-6, 3.5820749117985074`^-6, 1.7875592758145471`^-6, 1.2328068403994894`^-6, 7.293091860743975`^-7, 4.176729886963205`^-7, 2.3759152115189803`^-7, 1.059500993678597`^-7, 0.`}; b2 = Map[Round[Flatten[#], 0.001] &, b]; ListContourPlot[b2, ContourShading -> None, Mesh -> False, Contours -> t, Frame -> True, FrameLabel -> {"UTMeasting", "UTMnorting"}, LabelStyle -> Directive[FontSize -> 14, Bold], ImageSize -> 600] where the text file (+/- 400 mb) contains the list with the triplets. Unfortunately, rendering the plot seems to be impossible, even after waiting for more than 3 hours. A lot of memory is consumed, but that's it. I tried to exactly the same in Python and I got the plot in a few minutes, which made me somehow suspicious about what Mathematica is exactly doing. Let me know whether you have any experience with this. I can also provide the data file upon request.

I wanna create a ListContourPlot from a list with about 7000000 triplets (x,y,z) using the following piece of code:

a = ToExpression[Import["TIBBuil4totaal.txt", "List"]];
b = Map[Join[#[[1]], #[[2]]] &, Partition[
Riffle[Partition[Riffle[a[[All, 1]], a[[All, 2]]], 2], 
 Partition[a[[All, 3]], 1]], 2]];
t = {0.000051606234793646216`, 0.000013374424284682572`,    7.140323683145797`*^-6, 3.5820749117985074`*^-6,    1.7875592758145471`*^-6, 1.2328068403994894`*^-6,    7.293091860743975`*^-7, 4.176729886963205`*^-7,    2.3759152115189803`*^-7, 1.059500993678597`*^-7, 0.`};

b2 = Map[Round[Flatten[#], 0.001] &, b];

ListContourPlot[b2, ContourShading -> None, Mesh -> False,  Contours -> t, Frame -> True,   FrameLabel -> {"UTMeasting", "UTMnorting"},  LabelStyle -> Directive[FontSize -> 14, Bold], ImageSize -> 600]

where the text file (+/- 400 mb) contains the list with the triplets.

Unfortunately, rendering the plot seems to be impossible, even after waiting for more than 3 hours. A lot of memory is consumed, but that's it.

I tried to exactly the same in Python and I got the plot in a few minutes, which made me somehow suspicious about what Mathematica is exactly doing.

Let me know whether you have any experience with this. I can also provide the data file upon request.

POSTED BY: Jan Baetens

4 Replies

Sort By:

W. Craig Carter

W. Craig Carter, MIT

Posted 8 years ago

Hello, It is the rendering that is probably taking all the time. I don't have your data, so I am making a proxy problem: (size = 190)^3 a = Flatten[ Table[{x + y , y + z, z + x}, {x, size}, {y, size}, {z, size}], 2]; (a takes up \approx 0.9 GB) Timing[b = Map[Join[#[[1]], #[[2]]] &, Partition[ Riffle[Partition[Riffle[a[[All, 1]], a[[All, 2]]], 2], Partition[a[[All, 3]], 1]], 2]];] (~ 10 seconds and another 0.9GB, one could clear memory here if a is no longer needed) Timing[b2 = Map[Round[Flatten[#], 0.001] &, b];] (~3 seconds and about .2 GB) t = size Range[10]/10 Timing[lcp = ListContourPlot[b2, ContourShading -> None, Mesh -> False, Contours -> t, Frame -> True, FrameLabel -> {"UTMeasting", "UTMnorting"}, LabelStyle -> Directive[FontSize -> 14, Bold], ImageSize -> 600];] (~100 seconds and about .1GB) I didn't try to render lcp. I have a bunch of notebooks open and can't afford a crash. HTH, Craig

Hello, It is the rendering that is probably taking all the time.

I don't have your data, so I am making a proxy problem:

(size = 190)^3

a = Flatten[
   Table[{x + y , y + z, z + x}, {x, size}, {y, size}, {z, size}], 2];  (*a takes up \approx 0.9 GB*)

Timing[b = 
   Map[Join[#[[1]], #[[2]]] &, 
    Partition[
     Riffle[Partition[Riffle[a[[All, 1]], a[[All, 2]]], 2], 
      Partition[a[[All, 3]], 1]], 2]];]  (*~ 10 seconds and another 0.9GB, one could clear memory here if a is no longer needed*)



Timing[b2 = Map[Round[Flatten[#], 0.001] &, b];] (*~3 seconds and about .2 GB*)

t = size Range[10]/10

Timing[lcp = 
   ListContourPlot[b2, ContourShading -> None, Mesh -> False, 
    Contours -> t, Frame -> True, 
    FrameLabel -> {"UTMeasting", "UTMnorting"}, 
    LabelStyle -> Directive[FontSize -> 14, Bold], ImageSize -> 600];]  (*~100 seconds and about .1GB*)

I didn't try to render lcp. I have a bunch of notebooks open and can't afford a crash.

HTH, Craig

POSTED BY: W. Craig Carter

Jan Baetens

Jan Baetens, Ghent University

Posted 8 years ago

Running your snippet takes indeed as much time if you're indicating, but when I do a timing of my piece of code, written in exactly the same way as yours (so suppressing the graphical output), it does not halt....