Just a note, ResizeLayer has been updated to work on arbitrary rank
ResizeLayer
ResizeLayer[{1, 2, 3, 4, 5}][RandomReal[1, {3, 2, 2, 2, 2, 2}]] // Dimensions (* {3, 1, 2, 3, 4, 5} *)
Awesome! Time for some recoding then.