I'm trying to solve Maximum Entropy problems with Mathematica. The simplest one is a fair coin. It should solve this system of three equations:
S[p1_, p2_] := p1 Log[p1] + p2 Log[p2];
L[p1_, p2_, b_] := S[p1, p2] + b (1 - p1 - p2);
Solve[
{D[L[p1, p2, b], p1] == 0,
D[L[p1, p2, b], p2] == 0,
D[L[p1, p2, b], b] == 0},
{p1, p2, b}]
Mathematica correctly understands that this means solving this:
Solve[
{Log[p1] == b - 1,
Log[p2] == b - 1,
p1 + p2 == 1},
{p1, p2, b}]
But it tells me "This system cannot be solved with the methods available to Solve". Which I found amazing. So I tried this:
Solve[
{Log[p1] == Log[p2],
p1 + p2 == 1},
{p1, p2}]
which it solves correctly as p1-> 1/2 and p2->1/2
What is going on here??