Group Abstract Group Abstract

Message Boards Message Boards

Speed up the evaluating of a function?

Posted 9 years ago
POSTED BY: Alessio Lapolla
4 Replies

Alessio,

You have multiple syntax errors here.

List_ can't be a pattern (must be lower case or something else), Generally its a bad idea to capitalize your variables because you will interfere with Mathematica's symbols (as you do here).

Next time, please upload code that demonstrates your issue. Since you did not, I created some random data with 1 Million lists:

list2 = Table[RandomInteger[5, RandomInteger[{1, 10}]], 1000000];

Define the function:

f[lt_] := Product[1 - i, {i, lt}] 

Your approach:

AbsoluteTiming[Mean[Table[f[list2[[i]]], {i, 1, Length[list2]}]]]

7.2 seconds

Change to Map:

AbsoluteTiming[Mean[Map[f, list2]]]

4.25 seconds

Compile the function and Map it:

foo = Compile[{{lst, _Integer, 1}}, Product[1 - i, {i, lst}]]
AbsoluteTiming[Mean[Map[foo, list2]]]

0.45 seconds

Simple steps got a factor of 16 improvement. You likely could do even better by reworking the algorithm but this is a start...

Regards

POSTED BY: Neil Singer

Alessio,

The code you posted is more helpful. In this case MapThread sped things up by about 10%. Compile was 14 times faster than Map uncompiled and that is without reworking the algorithm to be faster. I really suggest you stop using Capital letters for your variables. One day they will get you in trouble.

s2c=Compile[{{y, _Integer},{d, _Integer},{p, _Real},{p1, _Real},{p2, _Real},{pp,_Real,1}},
       y (1-p-d/2*(Exp[-y] p1 p2)/(1-(1-Exp[-y]) p1 p2))+Log[Exp[-y]+(1-Exp[-y]) Product[1-j,{j,pp}]]-d/2 Log[1-(1-Exp[-y]) p1 p2]]

I created a test function since I do not know y, I just fixed y=1 and d.

smc[p_, p1_, p2_, pp_] := s2c[1, d, p, p1, p2, pp]

AbsoluteTiming[MapThread[smc, {p, p1, p2, pp}]]

Hope this helps

POSTED BY: Neil Singer

Dear Neil Thank you very much. You really helped me a lot. I will try to remember your hint: I will not use anymore the capital letters.

POSTED BY: Alessio Lapolla

Thank you for the answer. I did not post my code because it is pretty long and I tried to concentrate to the delicate point. It was my fault because your answer does not work in my case. Assuming that n=10^6. I have a list of this kind

a=RandomReal[{0,1},1000000];

then using this numbers I calculate these lists:

d = 4;
df = RandomVariate[PoissonDistribution[d], n];
p = RandomSample[a, n];
p1 = RandomSample[a, n];
p2 = RandomSample[a, n];
pp = Table[RandomSample[a, df[[i]]], {i, 1, n}];

as you can see pp is the irregular one the others are one level lists.

The function, that I need to evaluate, is (y is just a real number)

S2[y_, d_, p_, p1_, p2_, pp_] := 
 y (1 - p - d/2*(Exp[-y] p1 p2)/(1 - (1 - Exp[-y]) p1 p2)) + 
  Log[Exp[-y] + (1 - Exp[-y]) Product[1 - j, {j, pp}]] - 
  d/2 Log[1 - (1 - Exp[-y]) p1 p2]

At this point I calculated the million of contributions

 Table[S2[y, d, p[[i]], p1[[i]], p2[[i]], pp[[i]]], {i, 1, n}];

and then the mean (of course I can do two passage in one but the mean is not important here, I think). In this case I do not understand how can I use Map instead of Table. Then can you explain to me why Map is much more efficient than Table in this case?

POSTED BY: Alessio Lapolla
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard