This MacBook Pro has two cores, so your point about a function only using one core helps it make sense. Thanks for your help.

My MacBook Pro has 16 GB. Evaluating MaxMemoryUsed[] returns 402 903 496.

You have enough memory. Probably you have a CPU with 4 cores and MathKernel completely uses resources of one core (permanent CPU load is 25%). AFAIK the FindGeometricTransform function cannot use more than one core, the same is true for the most functions in Mathematica. The built-in Parallel* functionality can help if you can parallelize your task by yourself.

I'm still puzzled why the CPU load was so low; it never went above 30%.

How many physical memory do you have? The usual reason for low CPU load is insufficient physical memory which causes swapping. You can get the memory required for evaluation of your code by evaluating MaxMemoryUsed[] after the code. Note that the value returned includes only memory used by the Kernel and does not include the memory used by the FrontEnd.