WriteLine write each argument after the first on a new line in the file:
WriteLine[file, "Some text that is a question like Solve", RandomChoice[{1, 2, 3, 4, 5}], "+", RandomChoice[{1, 2, 3, 4, 5}]]
FilePrint["testQuestion.txt"]
Some text that is a question like Solve
3
+
5
StringForm is an easy way to get all the arguments composed into a single String:
StringForm["Solve `` + `` =", RandomChoice[{1, 2, 3, 4, 5}],
RandomChoice[{1, 2, 3, 4, 5}]]
Solve 4 + 3 =
Putting it all together (after closing and deleting the file):
Close[file];
DeleteFile[file]
file = File[CreateFile["testQuestion.txt"]];
WriteLine[file, StringForm["Solve `` + `` =", RandomChoice[{1, 2, 3, 4, 5}],
RandomChoice[{1, 2, 3, 4, 5}]]]
FilePrint[file]
Solve 2 + 5 =