Unsure if mathematica does this, but (true) OpenGL cards (likely any 3d card) support "sphere". That's not off topic because each sphere is a mesh generated by the video card (which infact can be generated per frame, or held as a set of points, even altered, and rotated). If you generate a ball mesh you'll quickly see you need about 100 vertices to make it a ball shape - more like 1000 to make it nice sphere appearance.
In Mathematica 4.0 (excuse me i'm more familiar with it), I find Sphere... PlotPoints->50 a believable ball shape, 2400 vertexes, so 50k balls is more like over 2.4 x 5 x 10^7, vertexes minimum, well over 100 million.
Below is 50,000 objects that rotates like butter. (use triangle shapes for more speed is a 3d lesson)
Graphics3D[
Table[Pyramid[{{Random[], 0, 0}, {0, Random[], 0}, {-Random[], 0,
0}, {0, -Random[], 0}, {0, 0, Random[]}} + i], {i, 1, 50000}]]
you'll need to run it at 50 to see they are actaully pyramids
at any rate: rotating 50k objects will not be same as rotating a mesh with 50k points - be sure of that. (old true all-in-silicon opengl did just as well with polygon as pyramids, but gaming cards today are not optimized that way and do much better with pyramids, last i read about it)