Thanks, Vitaliy. Yes, you would think that generalization of Shannon entropy would do the job, but I had a quick play with it and wasn't able to get a result. I probably just don't know what I'm doing, though.

Thanks, Henrik. Any example is helpful at the moment; it might help me to derive a general solution.

Simpler:

 p = \[Lambda]^k Exp[-\[Lambda]]/k!;
 -Sum[#, {k, 0, Infinity}] & /@ (p Log[p] // PowerExpand // Expand)

 (*\[Lambda] - \[Lambda]*Log[\[Lambda]] - Sum[-((\[Lambda]^k*Log[k!])/(E^\[Lambda]*k!)), {k, 0, Infinity}]*)

Test:

Integrate[Sin[x] + Cos[x] + Exp[-Pi*x^2 - 1/x], x](*Can't integrate all terms*)

but:

Integrate[#, x] & /@ (Sin[x] + Cos[x] + Exp[-Pi*x^2 - 1/x])
(*-Cos[x] + Integrate[E^(-x^(-1) - Pi*x^2), x] + Sin[x]*)

I successfully solved another pmf, but then I accidentally lost the equation after copying down the result. Now I can't recreate the result!

This is what I'm trying:

q = ((T - 1)/T)^i;
p = (1/T) q;
pmf = if[i = n, q, p];
-Sum[#, {n, 0, Infinity}] & /@ (pmf Log[pmf] // PowerExpand // Expand)

And I previously got a result of:

-((-1 + T) Log[-1 + T]) + T Log[T]

which I verified numerically against other data I have. But now when I run the above commands, I get a very ambiguous result that is hardly simplified at all.

What am I going wrong in trying to recreate the result? It seemed to easy the first time.

Thanks, Henrik, yes it took me a while before I realized that Mathematica uses = for assignment. Apologies for the half-baked plea for help, I did figure it out by myself in the end.

n is the size of the discrete random variable minus one, and i is the index you might say of the outcome, so when calculating numerical values of entropy for a drv of finite size, it computes essentially computes p[i] until i == n where it then computes q[i]. But my mathematics is not very strong and I didn't know how to express that in a way Mathematica would understand, so I actually just got rid of q altogether. The numerical results are spot on, but I'm not sure if the symbolic result is right, given that I dropped the case of when it reaches n (since n is Infinity in the analysis).

I also seemed to be having problems with variable declarations leaking from previous calculations. As in, the k variable from calculating Poisson still existed and so if I used k again in the definition of p then it didn't work. I can't explain in detail as I haven't had time to experiment, but essentially I couldn't run the Poisson calculation twice -- the second time would give me an error.

Finally I made the mistake of changing pmf to p in the Sum without removing the definition fo pmf and so the # symbol still referred to pmf although I didn't want it to. Rookie mistakes.

This is what I did in the end, but with a lot of Clear commands preceding it:

q = ((T - 1)/T)^i;
p = (1/T) q;
Simplify[-Sum[#, {i, 0, Infinity}] & /@ (p Log[p] // PowerExpand // Expand)]

Giving the aforementioned result:

-((-1 + T) Log[-1 + T]) + T Log[T]

Thanks for your help.

Thanks for the quick intro, Henrik!

Thanks, Mariusz. That question on the Mathematica overflow is closely related, and it helped me find other people asking the same question as me on there!