# Best optimization for maximum entropy?

**URL:** <https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978>\
**Category:** Questions\
**Tags:** optimization, numerical-computation, statistics-and-probability\
**Created:** [September 17, 2025, 9:03pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978 "2025-09-17T21:03:22Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![russell](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/russell/32/19742_2.png) [@russell](https://community.wolfram.com/u/russell)\
**Post date:** [September 17, 2025, 9:03pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/1 "2025-09-17T21:03:22Z")

</div>

Hello. I am attempting to implement the maximum entropy (“MaxEnt”) approach to species distribution modeling in the WL. The attached notebook has a function which calculates a penalized log-likelihood. It is based on the Excel workbook here: [https://www.ecography.org/appendix/e7872](https://www.ecography.org/appendix/e7872). It also has the example data.

My question is how best to optimize over the parameters, which in this case are in the form of a twelve element vector? There are now so many optimization routines and options that it is a bit overwhelming!

[Open in Wolfram Cloud ↗](https://www.wolframcloud.com/obj/9faaae11-15d9-4f3c-9bca-29f205ba810b)

---

<div class="post-metadata">

**Author:** ![russell](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/russell/32/19742_2.png) [@russell](https://community.wolfram.com/u/russell)\
**Post date:** [September 18, 2025, 1:59pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/2 "2025-09-18T13:59:23Z")

</div>

Attempting to answer my own question by some trial and error. First up: Wolfram is finding better optima than Excel! (No surprise there.)

[Open in Wolfram Cloud ↗](https://www.wolframcloud.com/obj/3349b8a3-e684-45b0-9800-c0b18f238328)

---

<div class="post-metadata">

**Author:** ![gianlucagorni](https://avatars.discourse-cdn.com/v4/letter/g/ec9cab/32.png) [@gianlucagorni](https://community.wolfram.com/u/gianlucagorni)\
**Post date:** [September 19, 2025, 3:34pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/3 "2025-09-19T15:34:18Z")

</div>

It seems to me (check it first) that instead of

```
expSum = Exp[sum - Max[sum]];
q = expSum/Total[expSum];
lnQ = Log[q];
logL = Total[Take[lnQ, n]]/n;

```

you can write

```
logL = Total[Take[sum, n]]/n - Log[Total[Exp[sum]]];

```

which is symbolically simpler. I don’t know if this helps with the optimization.

---

<div class="post-metadata">

**Author:** ![russell](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/russell/32/19742_2.png) [@russell](https://community.wolfram.com/u/russell)\
**Post date:** [September 19, 2025, 10:49pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/4 "2025-09-19T22:49:05Z")

</div>

OMG! It is _so_ much faster, presumably because, according to ByteCount, the symbolic expression is less than 4% the size of my original version.

[Open in Wolfram Cloud ↗](https://www.wolframcloud.com/obj/ca644381-eccb-4fe1-931f-cc42e902f2c9)

---

<div class="post-metadata">

**Author:** ![russell](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/russell/32/19742_2.png) [@russell](https://community.wolfram.com/u/russell)\
**Post date:** [September 19, 2025, 10:53pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/5 "2025-09-19T22:53:04Z")

</div>

It’s also a slightly better fit again. Still, I would like to know how it compares to a purely numerical fit to a compiled version of the code as given, since the main calculation is a matrix multiplication, which should be very fast. (Surely the symbolic expression generated internally by FindMaximum loses the matrix form?)

But thanks so much!

---

<div class="post-metadata">

**Author:** ![gianlucagorni](https://avatars.discourse-cdn.com/v4/letter/g/ec9cab/32.png) [@gianlucagorni](https://community.wolfram.com/u/gianlucagorni)\
**Post date:** [September 20, 2025, 10:57am UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/6 "2025-09-20T10:57:14Z")

</div>

If I am not mistaken, you had better get rid of the line

```
sum = sum - Max[sum]

```

because the `Max[sum]` term cancels out anyway in the next step (in theory). In practice it may survive and make calculations more complicated than necessary.

---

<div class="post-metadata">

**Author:** ![russell](https://yyz1.discourse-cdn.com/wolfram/user_avatar/community.wolfram.com/russell/32/19742_2.png) [@russell](https://community.wolfram.com/u/russell)\
**Post date:** [September 22, 2025, 1:59pm UTC](https://community.wolfram.com/t/best-optimization-for-maximum-entropy/25978/7 "2025-09-22T13:59:17Z")

</div>

Ha! You are right it doesn’t change the output, and removing it makes it 10x faster again. So the lesson here folks is improvements in the underlying function can make the biggest difference. In my defense I was following a template made by others, and now I have to figure out why, for example, the ‘divide by the max value’ step was in there. Maybe it is used later. But even if it is, it can be left out of the fitting process!

Having said this, I _still_ want to know how one would code this as a direct numerical optimization, ideally specifying the 12 parameters as a single vector (list). It is possible?