Message Boards Message Boards

Sum positive elements of a list efficiently?

GROUPS:

There are many ways to sum positive elements of a list such as, e.g.,

In[1]:= list = RandomReal[{-10, 10}, 1000000];

In[2]:= tList = {
  Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming
  }

Out[2]= {{2.51129, 2.49923*10^6}, {1.93032, 2.49923*10^6}, {1.6144, 2.49923*10^6}}

In[3]:= BarChart@tList[[All, 1]]

enter image description here

Can you recommend a code better than above ones?

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Total[Clip[list, {0, ∞}]] is pretty quick.

POSTED BY: J. M.
Answer
10 months ago

Fantastic! Thanks a lot! You are magician!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

I just wanted to reply this, Clip is the way to do this!

In the next release a new function will arrive that will cut this time in half (compared to Clip).

POSTED BY: Sander Huisman
Answer
10 months ago

It's very impressive for me!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

I believe I've seen that, and I have to agree. But let's not spoil things for other people. ;)

POSTED BY: J. M.
Answer
10 months ago

To be more specific:

Version 11 will feature Ramp that slices the timings of Clip in half:

http://reference.wolfram.com/language/ref/Ramp.html

POSTED BY: Sander Huisman
Answer
10 months ago

Thanks, Sander, for the information about function Ramp[]! We just have to wait for version 11 of Mma!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago
In[4]:= tList = {Plus @@ Map[If[# < 0, 0, #] &, list] // 
   AbsoluteTiming, 
  Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming, 
  Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
  Plus @@ Select[list, # > 0 &] // AbsoluteTiming }

Out[4]= {{0.942527, 2.49442*10^6}, {0.977524, 
  2.49442*10^6}, {0.778662, 2.49442*10^6}, {0.595495, 2.49442*10^6}}
POSTED BY: Frank Kampas
Answer
10 months ago

Thanks!

In[6]:= list = RandomReal[{-10, 10}, 1000000];

In[10]:= tList = {
Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
Total[Clip[list, {0, \[Infinity]}]] // AbsoluteTiming
}

Out[10]= {{1.76125, 2.49895*10^6}, {1.78087, 2.49895*10^6}, {1.42525,  2.49895*10^6},
{1.07893, 2.49895*10^6}, {0.0146056, 2.49895*10^6}}

In[11]:= BarChart@tList[[All, 1]]

enter image description here

Unbelievable fast!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

How about

Total[Clip[list, {0, Infinity}]]
POSTED BY: John Doty
Answer
10 months ago

Yes, it is the fastest! It is the lowest bar in the precedent BarChart!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

The function Clip[] is useful for multiplication, too...

In[1]:= list = RandomReal[{-10, 10}, 1000000];

In[6]:= tList = {
  Times @@ Map[If[# <= 0, 1, #] &, list] // AbsoluteTiming,
  Times @@ Select[list, # > 0 &] // AbsoluteTiming,
  Times @@ Clip[list, {0, \[Infinity]}, {1, \[Infinity]}] // AbsoluteTiming
  }

Out[6]= {{1.75065, 5.286174502515*10^282473}, {1.06879, 
  5.286174502515*10^282473}, {0.268463, 5.286174502515*10^282473}}

enter image description here

But, what about the case when the list contains not only numerical elements? Can be Clip[] used?

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Then how do you know if they're positive? How do you wish them to be dealt with?

POSTED BY: J. M.
Answer
10 months ago

It was a generic question, meaning that we need another approach... May be based on piece-wise functions...

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

That wasn't an answer to my question. :) If you have a list like {1, 2, x, -1, -4}, should x be added to the sum, or removed from the list before summing? Same question for {1, 2, I, -1, -4}.

POSTED BY: J. M.
Answer
10 months ago

Removed! List may contains elements of different types: chars, images and so on. But we want to sum only numbers. Ok. Positive ones! List may not contains numbers! In such case the result must be clear - that list doesn't contain numbers!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

And what about the second list I gave? Complex numbers like I are neither positive nor negative.

POSTED BY: J. M.
Answer
10 months ago

We can examine it as special case, i.e.in our case such numbers are not summed.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

As you asked me, J.M., about the summation, we can imagine an improvement of the function Total[] that will include the criterion according to which the summation is done, i.e.

Total[list, crit, {Subscript[n, 1],Subscript[n, 2]}]

crit means criterion and it may be, e.g., Reals, Complex, Integers, and so on...

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Then the use of Select[]/Cases[]/Pick[] can prolly not be avoided. Use e.g. Total[Select[list, Positive]].

POSTED BY: J. M.
Answer
10 months ago

Thanks a lot, J.M.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Another approach is:

AbsoluteTiming[pcf = Piecewise[{{x, x > 0}, {0, True}}]; Total[pcf /@ list];]

But it is slower (at least on my computer).

POSTED BY: Sander Huisman
Answer
10 months ago

I'd have done it this way: pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable]; Total[pcf[tst]]

POSTED BY: J. M.
Answer
10 months ago

@Sander The code contains errors! Try it on a small list!

After J.M. improvement

In[32]:= list = RandomReal[{-10, 10}, 1000000];

In[33]:= tList = {
  Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
  Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
  Total[Clip[list, {0, \[Infinity]}]] // AbsoluteTiming,
  AbsoluteTiming[
   pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable]; 
   Total[pcf[list]]]
  }

Out[33]= {{1.71495, 2.50376*10^6}, {1.70818, 2.50376*10^6}, {1.3847, 
  2.50376*10^6}, {1.03566, 2.50376*10^6}, {0.0151026, 
  2.50376*10^6}, {3.1214, 2.50376*10^6}}

In[34]:= BarChart@tList[[All, 1]]

enter image description here

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

You are right, I copied the wrong stuff from my test-notebook... But fixed by JM.. thanks

POSTED BY: Sander Huisman
Answer
10 months ago

At the moment, there are 8 pieces of code. They are sorted from the fastest to the slowest:

In[47]:= list = RandomReal[{-10, 10}, 1000000];

In[48]:= tList = {
  (*1*) Total[Clip[list, {0, \[Infinity]}]] // AbsoluteTiming,
  (*2*) Total[Select[list, Positive]] // AbsoluteTiming,
  (*3*) Total[Cases[list, x_ /; x > 0]] // AbsoluteTiming,
  (*4*) Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
  (*5*) Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
  (*6*) Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (*7*) Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (*8*) (pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable]; 
    Total[pcf[list]]) // AbsoluteTiming
  }

Out[48]= {{0.0181547, 2.50798*10^6}, {0.656519, 
  2.50798*10^6}, {0.988164, 2.50798*10^6}, {1.0865, 
  2.50798*10^6}, {1.41506, 2.50798*10^6}, {1.77652, 
  2.50798*10^6}, {1.77717, 2.50798*10^6}, {3.21233, 2.50798*10^6}}

enter image description here

The function Total[] is the key to the fast codes. Total[] together with Clip[] form the fastest code:

Total[ Clip[list, {0, $\infty$}] ]

It is 36 times faster than the second code, and 177 times faster than the last!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

You may want to use RepeatedTiming at least for the first one, because it is so fast! On my machine just 2.8 milliseconds!

POSTED BY: Sander Huisman
Answer
10 months ago

Sure, I can do it, but for our goals it's not so important! It's just for principles and ideas.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

In short: use the thing with Select[] if you are expecting anything that isn't a real number in your list (which you can test with FreeQ[]). Otherwise, use the one with Clip[].

POSTED BY: J. M.
Answer
10 months ago

Yes! Simple and Clear!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

I had used formerly Procedural Programming a lot. So, I decided to test that approach and added 4 pieces of code based on the functions: Do, For, While and Table. The results are beneath.

In[55]:= list = RandomReal[{-10, 10}, 1000000];

In[56]:= tList = {
  (* 1*) Total[Clip[list, {0, \[Infinity]}]] // AbsoluteTiming,
  (* 2*) Total[Select[list, Positive]] // AbsoluteTiming,
  (* 3*) Total[Cases[list, x_ /; x > 0]] // AbsoluteTiming,
  (* 4*) Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
  (* 5*) Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
  (* 6*) Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (* 7*) Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (* 8*)(s = 0; l = Length[list]; 
    Do[If[list[[i]] > 0, s += list[[i]], s], {i, l}]; s) // 
   AbsoluteTiming,
  (* 9*)(s = 0; l = Length[list]; 
    Table[If[list[[i]] > 0, s += list[[i]], s], {i, l}][[l]]) // 
   AbsoluteTiming,
  (*10*)(pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable]; 
    Total[pcf[list]]) // AbsoluteTiming,
  (*11*)(s = 0; l = Length[list]; 
    For[i = 0, i <= l, i++, If[list[[i]] > 0, s += list[[i]], s]]; s) //
    AbsoluteTiming,
  (*12*)(s = 0; i = 1; l = Length[list]; 
    While[i <= l, If[list[[i]] > 0, s += list[[i]], s]; i++]; s) // 
   AbsoluteTiming
  }

Out[56]= {{0.0173669, 2.50005*10^6}, {0.627804, 
  2.50005*10^6}, {0.953728, 2.50005*10^6}, {1.16986, 
  2.50005*10^6}, {1.51738, 2.50005*10^6}, {1.7455, 
  2.50005*10^6}, {1.74073, 2.50005*10^6}, {2.89995, 
  2.50005*10^6}, {2.91044, 2.50005*10^6}, {3.53485, 
  2.50005*10^6}, {3.74655, 2.50004*10^6}, {3.80228, 2.50004*10^6}}

In[57]:= BarChart@tList[[All, 1]]

enter image description here

The code with While[] is 219 slower than the fastest code! Comments are superfluous!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Bingo?

I found that for small lists the code Total[Clip[list, {0, [Infinity]}]] is not more the fastest. The fastest is a new piece of code

Plus @@ Clip[list, {0, \[Infinity]}]

but only for lists with small dimensions ( = 20). I have done experiments with the following pieces of code

In[428]:= tList = {
  (* 1*) Total[Clip[list, {0, \[Infinity]}]] // AbsoluteTiming,
  (* 2*) Plus @@ Clip[list, {0, \[Infinity]}] // AbsoluteTiming,
  (* 3*) Total[Select[list, Positive]] // AbsoluteTiming,
  (* 4*) Total[Cases[list, x_ /; x > 0]] // AbsoluteTiming,
  (* 5*) Plus @@ Cases[list, x_ /; x > 0] // AbsoluteTiming,
  (* 6*) Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
  (* 7*) Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
  (* 8*) Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (* 9*) Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (*10*)(s = 0; l = Length[list]; 
    Do[If[list[[i]] > 0, s += list[[i]], s], {i, l}]; s) // 
   AbsoluteTiming,
  (*11*)(s = 0; l = Length[list]; 
    Table[If[list[[i]] > 0, s += list[[i]], s], {i, l}][[l]]) // 
   AbsoluteTiming,
  (*12*)(pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable]; 
    Total[pcf[list]]) // AbsoluteTiming,
  (*13*)(s = 0; l = Length[list]; 
    For[i = 0, i <= l, i++, If[list[[i]] > 0, s += list[[i]], s]]; 
    s) // AbsoluteTiming,
  (*14*)(s = 0; i = 1; l = Length[list]; 
    While[i <= l, If[list[[i]] > 0, s += list[[i]], s]; i++]; s) // 
   AbsoluteTiming
  }

Out[428]= {{0.0000371432, 64.4051}, {0.0000244363, 
  64.4051}, {0.0000288349, 64.4051}, {0.0000483839, 
  64.4051}, {0.0000317672, 64.4051}, {0.0000288349, 
  64.4051}, {0.0000562036, 64.4051}, {0.0000498501, 
  64.4051}, {0.0000381207, 64.4051}, {0.0000659781, 
  64.4051}, {0.0000596246, 64.4051}, {0.0000850384, 
  64.4051}, {0.0000762413, 64.4051}, {0.0000772188, 64.4051}}

I observed that if the experiments are repeated, the time results are not the same. You can view this fact in the following animation, which includes 120 repetitions of the same summations on the same list.

enter image description here

A question appears: why the times are different if the code and the list remain unchanged?

I have done the same experiment on the list with 1 mln elements, but providing repetitions only 32 times. The results may be seen in the following GIF

enter image description here

For the large lists, the time behavior of the pieces of code are more stable.

Nevertheless, one more question appears: what is the worst correct piece of code?

May be the following code a pretender to such denomination?

Total[list //. {a___, b_, c___} /; b < 0 -> {a, 0, 
     c}] // AbsoluteTiming

For small lists it works perfect! But for lists with length more than 1000 it requires very long time.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

"I observed that if the experiments are repeated, the time results are not the same." - that's precisely why Sander was telling you to use RepeatedTiming[] instead, upthread. It should average out any time differences across different runs.

Also, you might consider using a fixed PlotRange on your charts, so that the resulting animation does not look jittery.

POSTED BY: J. M.
Answer
10 months ago

I have used RepeatedTiming[] instead of AbsoluteTiming[] and I have repeated summations 120 times for a list with 20 elements. A have used a fixed PlotRange[]. So, the results are these

enter image description here

With RepeatedTiming[] the fastest is still Total[Clip[list, {0, [Infinity]}]], but the difference is not so essential.

For the list with 1 mln elements, I have done 10 summations

enter image description here

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Nevertheless, J.M., why is so slow the following code?

Total[list //. {a___, b_, c___} /; b < 0 -> {a, 0, c}]

Must it be so slow because of its essence?

Does the expression

{a___, b_, c___} /; b < 0 -> {a, 0, c}

mean simply a substitution of negative elements by 0 or the expression has some other meaning?

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Repeatedly matching the entire list is indeed very likely to be slow, so why do that here? You could do Total[list /. b_?Negative -> 0] if you wish to take the replacement route.

POSTED BY: J. M.
Answer
10 months ago

This code doesn't work.

In[12]:= list = {-1, 2, 1, 2, -1};
Total[list /. Negative[b] -> 0]
Out[13]= 3

But, I have such code with replacement in my test list of codes and it works. It's somewhat strange in my opinion that Repeatedly is not optimized to recognize such simple problems.

Thanks, J.M. May be I will return with mixed list formed both of numerical and other types of elements.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Sorry 'bout that; I've edited the post. I seem to have pasted the wrong snippet.

POSTED BY: J. M.
Answer
10 months ago

Dear J.M., any response of yours produces improvement of the entire list of codes. I entered your code in the list after the number 7 marked by ' sign. I have changed verification of the expressions like ">0" or "<0" by checking ?Positive or ?Negative in 2 of codes. So, I have obtained two new modified pieces of code after numbers 4 and 5.

First, I have done tests on a list with 1 mln list elements and with AbsoluteTiming 48 repetitions.

tList = {
  (*  1 *) Total[Clip[list, {0, \[Infinity]}]] // AbsoluteTiming,
  (*  2 *) Plus @@ Clip[list, {0, \[Infinity]}] // AbsoluteTiming,
  (*  3 *) Total[Select[list, Positive]] // AbsoluteTiming,
  (*  4 *) Total[Cases[list, x_ /; x > 0]] // AbsoluteTiming,
  (*   '*) Total[Cases[list, _?Positive]] // AbsoluteTiming,
  (*  5 *) Plus @@ Cases[list, x_ /; x > 0] // AbsoluteTiming,
  (*   '*) Plus @@ Cases[list, _?Positive] // AbsoluteTiming,
  (*  6 *) Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
  (*  7 *) Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
  (*   '*) Total[list /. b_?Negative -> 0] // AbsoluteTiming,
  (*  8 *) Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (*  9 *) Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
  (* 10 *)(s = 0; l = Length[list]; 
    Do[If[list[[i]] > 0, s += list[[i]], s], {i, l}]; s) // 
   AbsoluteTiming,
  (* 11 *)(s = 0; l = Length[list]; 
    Table[If[list[[i]] > 0, s += list[[i]], s], {i, l}][[l]]) // 
   AbsoluteTiming,
  (* 12 *)(pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable]; 
    Total[pcf[list]]) // AbsoluteTiming,
  (* 13 *)(s = 0; l = Length[list]; 
    For[i = 0, i <= l, i++, If[list[[i]] > 0, s += list[[i]], s]]; 
    s) // AbsoluteTiming,
  (* 14 *)(s = 0; i = 1; l = Length[list]; 
    While[i <= l, If[list[[i]] > 0, s += list[[i]], s]; i++]; s) // 
   AbsoluteTiming
  }

Animated results are in the following GIF

enter image description here

For the same list with 1 mln elements, the 24 tests with RepeatedTiming are reflected by the following GIF

enter image description here

For a list with 20 elements, the results of the 120 tests with AbsoluteTiming are reflected by the GIF:

enter image description here

For the same list with 20 elements, the results of the 120 tests with RepeatedTiming are reflected by the GIF:

enter image description here

Conclusion: making an efficient code needs to know some small and very important details. Some of them are inserted in the above list of codes.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

After the release of Mma 11, I introduced, according with Sander suggestion, two new codes with the function Ramp[].

The codes are sorted to emphasize their efficiency.

At the moment, there are 19 codes.

List with 1 mln random generated real elements

First, I have done tests on a list with 1 mln numeric elements

list = RandomReal[{-10, 10}, 1000000];

and AbsoluteTiming 48 repetitions:

ta = Table[tList = {
     (*  1 *) Total@Ramp@list // AbsoluteTiming,
     (*  2 *) Total@Clip[list, {0, \[Infinity]}] // AbsoluteTiming,
     (*  3 *) Plus @@ Ramp@list // AbsoluteTiming,
     (*  4 *) Plus @@ Clip[list, {0, \[Infinity]}] // AbsoluteTiming,
     (*  5 *) Total@Select[list, Positive] // AbsoluteTiming,
     (*  6 *) Total@Cases[list, _?Positive] // AbsoluteTiming,
     (*  7 *) Plus @@ Cases[list, _?Positive] // AbsoluteTiming,
     (*  8 *) Total@Cases[list, x_ /; x > 0] // AbsoluteTiming,
     (*  9 *) Plus @@ Cases[list, x_ /; x > 0] // AbsoluteTiming,
     (* 10 *) Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
     (* 11 *) Total[list /. b_?Negative -> 0] // AbsoluteTiming,
     (* 12 *) Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
     (* 13 *) Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
     (* 14 *) Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
     (* 15 *)(s = 0; l = Length[list]; 
       Do[If[list[[i]] > 0, s += list[[i]], s], {i, l}]; s) // 
      AbsoluteTiming,
     (* 16 *)(s = 0; l = Length[list]; 
       Table[If[list[[i]] > 0, s += list[[i]], s], {i, l}][[l]]) // 
      AbsoluteTiming,
     (* 17 *)(pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable];
        Total[pcf[list]]) // AbsoluteTiming,
     (* 18 *)(s = 0; l = Length[list]; 
       For[i = 0, i <= l, i++, If[list[[i]] > 0, s += list[[i]], s]]; 
       s) // AbsoluteTiming,
     (* 19 *)(s = 0; i = 1; l = Length[list]; 
       While[i <= l, If[list[[i]] > 0, s += list[[i]], s]; i++]; s) //
       AbsoluteTiming
     };
   BarChart[tList[[All, 1]], PlotRange -> {{-.01, 19.5}, {0, 4}}]
   , {48}];

enter image description here

Time results for the last 48th test:

In[1]:= AbsTiming = tList[[All, 1]]

Out[1]= {0.00773412, 0.0147376, 0.273374, 0.285654, 0.659804, \
0.799176, 0.794375, 0.980972, 0.982004, 1.07851, 1.22457, 1.39598, \
1.75444, 1.76957, 2.60787, 2.97176, 3.15208, 3.39943, 3.73759}

In[2]:= AbsTiming/AbsTiming[[1]]

Out[2]= {1., 1.90553, 35.3465, 36.9342, 85.3108, 103.331, 102.711, \
126.837, 126.97, 139.448, 158.333, 180.496, 226.844, 228.8, 337.19, \
384.241, 407.556, 439.537, 483.26}

For the same list with 1 mln elements, the results of 48 tests with RepeatedTiming in one GIF

enter image description here

The time results of the last 48th test:

In[3]:= RepTiming = tList[[All, 1]]

Out[3]= {0.0076, 0.015, 0.291, 0.294, 0.708, 0.8480, 0.843, 1.02, \
1.04, 1.2, 1.3, 1.5, 1.84, 1.85, 2.87, 3.13, 3.39, 3.7516, 4.01}

In[4]:= RepTiming/RepTiming[[1]]

Out[4]= {1.0, 2.0, 38., 39., 9.*10^1, 1.1*10^2, 1.1*10^2, 1.3*10^2, 
 1.4*10^2, 1.5*10^2, 1.8*10^2, 2.0*10^2, 2.4*10^2, 2.4*10^2, 3.8*10^2,
  4.1*10^2, 4.5*10^2, 4.9*10^2, 5.3*10^2}

Remark, the first code is 2 times faster than the second code and 530 times faster than the 19th code.

Short list with 20 random generated real elements

Third, I have done tests for a short list with 20 elements and with 120 repetitions with AbsoluteTiming[]

enter image description here

Time results for the last 120th test:

In[5]:= absTiming = tList[[All, 1]]

Out[5]= {0.0000112407, 0.0000156393, 0.0000136844, 0.0000141731, \
0.0000244364, 0.0000263913, 0.0000205266, 0.0000293237, 0.0000239477, \
0.0000259026, 0.0000498502, 0.0000474066, 0.000044963, 0.0000390982, \
0.0000606022, 0.000061091, 0.0000752641, 0.000077219, 0.0000767303}

In[6]:= absTiming/absTiming[[1]]

Out[6]= {1., 1.3913, 1.21739, 1.26087, 2.17391, 2.34783, 1.82609, \
2.6087, 2.13043, 2.30435, 4.43478, 4.21739, 4., 3.47826, 5.3913, \
5.43478, 6.69565, 6.86957, 6.82609}

Fourth, I have done tests with on the same short list with 20 elements with 120 repetitions with RepeatedTiming

enter image description here

Time results for the last 120th test:

In[7]:= repTiming = tList[[All, 1]]

Out[7]= {2.016*10^-6, 5.33*10^-6, 
 7.16*10^-6, 0.0000110, 0.000019, 0.00002111, 0.0000189, 0.0000249, \
0.00002229, 0.0000230, 0.0000373, 0.0000408, 0.0000387, 0.0000360, \
0.0000532, 0.0000568, 0.0000664, 0.0000718, 0.00009}

In[8]:= repTiming/repTiming[[1]]

Out[8]= {1.000, 2.64, 3.55, 5.48, 9.2, 10.47, 9.4, 12.3, 11.06, \
11.4, 18.5, 20.2, 19.2, 17.8, 26.4, 28.2, 32.9, 35.6, 4.*10^1}

Conclusions

The function Ramp[] together with Total[] is more faster than all the rest of codes as for large lists as for short lists.

These results may serve as a good bases to construct efficient summations in other situations and cases.

These codes are good to use in classrooms as examples of good/efficient and bad/non-efficient programming.

POSTED BY: Valeriu Ungureanu
Answer
10 months ago

Thanks for sharing!

POSTED BY: Sander Huisman
Answer
10 months ago

Thank you all for your contributions!

POSTED BY: Valeriu Ungureanu
Answer
10 months ago
(Total[Abs[list]] + Total[list])/2

This seems to beat all other timings on my machine (M 11).

POSTED BY: Douglas Kubler
Answer
9 months ago

Smart! Very nice find! Though compromises readability...

POSTED BY: Sander Huisman
Answer
9 months ago

For 10^6 items, the Ramp method is faster on my machine (Mac OSX V11) (using RepeatedTiming to get an accurate timing):

list = RandomReal[{-10, 10}, 1000000];
Total@Ramp@list // RepeatedTiming
(Total[Abs[list]] + Total[list])/2 // RepeatedTiming

{0.0014, 2.50381*10^6}
{0.0019, 2.50381*10^6}
POSTED BY: Sander Huisman
Answer
9 months ago

Readability is subject to Mathematica syntax, but if one understands the plan it seems clear. (and doesn't require M11)

shows lists aligned

Testing on HP5820, Windows 7 -

In[72]:= list = RandomReal[{-10, 10}, 1000000];
Total@Ramp@list // RepeatedTiming
(Total[Abs[list]] + Total[list])/2 // RepeatedTiming

Out[73]= {0.0026, 2.49858*10^6}

Out[74]= {0.00248, 2.49858*10^6}
POSTED BY: Douglas Kubler
Answer
9 months ago

Dear Douglas, Thank you for an impressive code, both very clever and simple. It’s amazing to discover once again that nice things may be done on the basis of very simple and known functions, like these Total[] and Abs[].

Your code together with the above codes are as a challenge for the new fastest code proposal. It will be very amazing if such code or codes will be find.

I am doing now the new tests which include your code too. I will display the results as soon as possible .

POSTED BY: Valeriu Ungureanu
Answer
9 months ago
list = RandomReal[{-10, 10}, 1000000];

List of codes

ta = Table[taList = {
     (*  1 *)(Total[Abs[list]] + Total[list])/2 // AbsoluteTiming,
     (*  2 *) Total@Ramp@list // AbsoluteTiming,
     (*  3 *) Total@Clip[list, {0, \[Infinity]}] // AbsoluteTiming,
     (*  4 *) Plus @@ Ramp@list // AbsoluteTiming,
     (*  5 *) Plus @@ Clip[list, {0, \[Infinity]}] // AbsoluteTiming,
     (*  6 *) Total@Select[list, Positive] // AbsoluteTiming,
     (*  7 *) Total@Cases[list, _?Positive] // AbsoluteTiming,
     (*  8 *) Plus @@ Cases[list, _?Positive] // AbsoluteTiming,
     (*  9 *) Total@Cases[list, x_ /; x > 0] // AbsoluteTiming,
     (* 10 *) Plus @@ Cases[list, x_ /; x > 0] // AbsoluteTiming,
     (* 11 *) Plus @@ Select[list, # > 0 &] // AbsoluteTiming,
     (* 12 *) Total[list /. b_?Negative -> 0] // AbsoluteTiming,
     (* 13 *) Total[list /. x_ /; x < 0 -> 0] // AbsoluteTiming,
     (* 14 *) Total@Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
     (* 15 *) Plus @@ Map[If[# < 0, 0, #] &, list] // AbsoluteTiming,
     (* 16 *)(s = 0; l = Length[list]; 
       Do[If[list[[i]] > 0, s += list[[i]], s], {i, l}]; s) // 
      AbsoluteTiming,
     (* 17 *)(s = 0; l = Length[list]; 
       Table[If[list[[i]] > 0, s += list[[i]], s], {i, l}][[l]]) // 
      AbsoluteTiming,
     (* 18 *)(pcf = Function[x, Piecewise[{{x, x > 0}}, 0], Listable];
        Total[pcf[list]]) // AbsoluteTiming,
     (* 19 *)(s = 0; l = Length[list]; 
       For[i = 0, i <= l, i++, If[list[[i]] > 0, s += list[[i]], s]]; 
       s) // AbsoluteTiming,
     (* 20 *)(s = 0; i = 1; l = Length[list]; 
       While[i <= l, If[list[[i]] > 0, s += list[[i]], s]; i++]; s) //
       AbsoluteTiming
     };
   BarChart[taList[[All, 1]], PlotRange -> {{-.01, 20.5}, {0, 4}}]
   , {48}];

Test Results for a list with 1mln elements

List generation:

list = RandomReal[{-10, 10}, 1000000];

Animation includes the results of 48 tests made both with AbsoluteTiming and RepeatedTiming. The first BarChart is for AbsoluteTiming.

enter image description here

Numerically, we can conclude that the best code is the newest:

Comparing results

In[102]:= absTiming = taList[[All, 1]]

Out[102]= {0.00687931, 0.00704303, 0.015065, 0.359983, 0.321751, 0.683816, 0.779643, \
0.887773, 1.03672, 1.0153, 1.14582, 1.31585, 1.45897, 1.83022, 1.79794, \
2.67337, 2.94904, 3.10739, 3.46186, 3.73936}

In[103]:= absTiming/absTiming[[1]]

Out[103]= {1., 1.0238, 2.1899, 52.3284, 46.7708, 99.4018, 113.332, 129.05, 150.701, \
147.587, 166.561, 191.277, 212.081, 266.048, 261.355, 388.611, 428.682, \
451.701, 503.228, 543.567}

In[108]:= repTiming = trList[[All, 1]]

Out[108]= {0.00618, 0.00703, 0.0147, 0.27, 0.271, 0.638, 0.7696, 0.775, 0.9548, 0.9669, \
1.09, 1.20, 1.368, 1.73, 1.75, 2.60, 2.86, 3.06, 3.5, 3.74}

In[109]:= repTiming/repTiming[[1]]

Out[109]= {1.00, 1.14, 2.38, 44., 43.9, 103., 125., 125., 155., 157., 176., 194., 222., \
280., 283., 421., 463., 495., 5.6*10^2, 6.1*10^2}

Test Results for a list with 20 elements

List generation:

list = RandomReal[{-10, 10}, 20];

Animation includes the results of 60 tests made both with AbsoluteTiming and RepeatedTiming. The first BarChart is for AbsoluteTiming.

enter image description here

Comparing results

In[95]:= abs20Timing = taList20[[All, 1]]

Out[95]= {0.0000166167, 
 4.39853*10^-6, 0.0000151505, 0.0000131956, 0.0000146618, 0.0000244363, \
0.0000268799, 0.0000219927, 0.0000298123, 0.0000254138, 0.0000268799, \
0.0000508275, 0.0000478952, 0.0000444741, 0.0000390981, 0.0000669555, \
0.0000645118, 0.0000772187, 0.0000825947, 0.0000796623}

In[96]:= abs20Timing/abs20Timing[[1]]

Out[96]= {1., 0.264706, 0.911765, 0.794118, 0.882353, 1.47059, 1.61765, 1.32353, \
1.79412, 1.52941, 1.61765, 3.05882, 2.88235, 2.67647, 2.35294, 4.02941, \
3.88235, 4.64706, 4.97059, 4.79412}

In[97]:= rep20Timing = trList20[[All, 1]]

Out[97]= {4.761*10^-6, 2.2*10^-6, 5.53*10^-6, 
 7.6*10^-6, 0.000011419, 0.0000173, 0.000026, 0.00002, 0.000030, 0.000026, \
0.000028, 0.000044, 0.000048, 0.000047, 0.0000413, 0.000073, 0.000069, 0.003, \
0.0001, 0.0000795}

In[98]:= rep20Timing/rep20Timing[[1]]

Out[98]= {1.000, 0.45, 1.16, 1.6, 2.398, 3.62, 5.5, 5., 6.3, 5.5, 5.9, 9.2, 10., 
 1.*10^1, 8.67, 15., 14., 7.*10^2, 2.*10^1, 16.7}

For the short list the code with Total[] and Ramp[] functions is the fastest.

POSTED BY: Valeriu Ungureanu
Answer
9 months ago

Group Abstract Group Abstract