multiple things could be the reason, I can only speculate:

Depending on the length of the input there might be different heuristics of what to do with it. e.g. if it is short, it is more probably to be (say) an animal name, then when it is a very long input. Perhaps a similar construct is in place for equations.

Also, there might be a construct in place that makes sure that (very long) input does not crash or unnecessarily overloads the system.

Perhaps you can run it in the Wolfram Cloud or (if available) on your own computer using Mathematica.

what about 25 or 30? i should have mentioned earlier, i did get inconstant results, sometimes it would accept 24, but not 25, sometimes it would accept 23 but not 24 etc

Here is something a little bit more systematic. The following code generates Wolfram|Alpha queries with different numbers of "a's".

results = 
  Monitor[Table[{i + 1, First@(List @@ WolframAlpha[Evaluate@StringJoin[Join[{"solve b=a"}, Table["+a", {i}], {" for a"}]], {{"Result", 1}, 
 "FormulaData"}])}, {i, 1, 40}], i];

It turns out that the interesting one is at the transition at 24 a's. The following runs 40 trials of that one request.

results2 = 
  Monitor[Table[{i,First@(List @@ WolframAlpha[Evaluate@StringJoin[Join[{"solve b=a"}, Table["+a", {23}], {" for a"}]], {{"Result", 1}, 
  "FormulaData"}])}, {i, 1, 40}], i];

Here are the results in two tables:

TableForm[{{Grid[Join[{{"Number a's", "Result"}}, results], Frame -> All], Grid[Join[{{"Trial", "Result"}}, results2], Frame -> All]}}]

Cheers,

Marco