A thing I am missing from psfrag is that the font size is no longer determined at complie time - no way MaTeX could know how I will rescale the image afterwards
This is a very good point. So many people get this wrong when creating figuresthey don't think about the final size of the image, and the fonts won't match the surrounding text after rescaling.
The MaTeX documentation includes a tutorial on how to deal with this. There are no magic solutions, unfortunately. It simply gives suggestions on how to plan ahead and consider the target figure size from the beginning, and how to use scaling and non-scaling units in Mathematica. (Note that all of them will scale once you've exported to PDFbut not when resizing within Mathematica.)
It's actually useful to plan ahead even if the font size could be determined at LaTeX-compile-time. It's not just the font sizes that should ideally not scale, but also some other features like point sizes and line widths. E.g., it's hard to judge the colour of a line if it's too thin. I like to specify line widths in absolute units (i.e. non-scaling units) for this reason.