Will be very difficult to correctly export this figure correctly, too many different substructures that can all have their issues.

Make one Grid is much easier

array = Prepend[Transpose[Prepend[Correlation[Most /@ g[[1]]], h]], 
   Prepend[h, ""]];
Grid[Map[
  If[NumberQ[#],
    Item[Style[NumberForm[#, {3, 2}], 11], 
     Background -> ColorData["Temperature"][#]],
    Item[Style[#, Bold, 11], Background -> White]
    ] &
  , array, {2}],
 Frame -> {All, 
   All, {{1, 1} -> False, {1, 2} -> True, {2, 1} -> True}},
 ItemSize -> {4, 4}, Spacings -> {0, 0}, 
 Alignment -> {Center, Center}]

Do you need the row and column labels to be part of the grid? Rather than using Row, Column, Overlay, GraphicsGrid, ..., consider using MatrixPlot with Epilog and custom Ticks. Take a look at this for an example.