I'd get a 27 inch iMac and the (as yet unreleased) thunderbolt accessory to add a NVIDIA GPU. The price estimates indicate that the whole package will come in at under $5K.

The screen is gorgeous.

The soon to be released iMac Pro starts at just under your limit, and has 8 cores instead of 4, but most of the libraries for GPU acceleration in Mathematica require NVIDIA, so the external GPU will put you over the limit.

There is always the chance that Wolfram will add support for AMD GPUs or support Metal (Metal 2), in which case a Stock iMac with an i7 chip would be fine.

I am sure that there are fine Linux boxes that could be assembled for less, but the screen you might get would not match the 5K resolution for the money. There also seem to be fewer annoyances run-in Mathematica on macOS than with either Windows or Linux.