Thank you so much Roland! You're absolutely right, that ended up being the problem. I found a note about it at the bottom of mathematica's documentation for NMaximize (I feel like an idiot for not finding it earlier), and there they offer a solution, which was just checking whether the passed objects into AffineFunction were numbers. So the fixed code just has the following definition for AffineFunction

affineFunction[a_?NumericQ, b_?NumericQ, c_?NumericQ, d_?NumericQ] :=

This seems to have worked.