@Marco Thiel thank you for the references and insight! I believe FindDistribution, while finding correct distributions, needs some tune up to get the distribution parameters right. To test the waters, I tried using EstimatedDistribution and got a bit better results. Let's run this for 500,000 points:
pts=RandomPoint[Disk[],500000];
mesh=VoronoiMesh[pts,PerformanceGoal->"Speed"];
vor=MeshPrimitives[mesh,2]
vorInner=Select[vor,RegionWithin[Disk[],#]&];
areas=Area/@vorInner;
hist=Histogram[areas,200,"PDF",PlotTheme->"Detailed"];
and then find the parameters assuming distributions are known:
disGAM = EstimatedDistribution[areas, GammaDistribution[a, b]]
GammaDistribution[3.526166220777205, 1.7809806247800413*^-6]
disEXT = EstimatedDistribution[areas, ExtremeValueDistribution[a, b]]
ExtremeValueDistribution[4.775094275466527*^-6, 2.5746412541824736*^-6]
We see gamma (red) behaves a bit better now, both for zero and for the tail values:
Show[hist,Plot[{PDF[disEXT,x],PDF[disGAM,x]},{x,0,.00002},PlotStyle->{Blue,Red}]]

But while p-value of gamma is greater than extreme, it is still pretty low, not sure if there is a catch here somewhere:
DistributionFitTest[areas,disGAM]
0.000051399258187534436`
DistributionFitTest[areas,disEXT]
0.`