Since Wolfram is investing in Machine learning (as I understand using MXNet) and using this nice Wolfram technology might result in some ML production projects with or without Wolfram technology as a follow-up, a decision must be made on what hardware infra to choose. Hewlett Packard created a project (by the HPE Labs) on github that can be used to test several setups.

Deep Learning Benchmarking Suite Deep Learning Benchmarking Suite (DLBS) is a set of command line tools for providing consistent and reproducible benchmark experiments on various hardware/software combinations. In particular, DLBS provides the following functionality:

Implements internally various deep models. Our goal is to provide same model implementations for all supported frameworks. Deep models that are supported include various VGGs, ResNets, AlexNet and GoogleNet models. Benchmarks single node multi GPU configurations. Frameworks that are now supported: BVLC Caffe, NVIDIA Caffe, Intel Caffe, Caffe2, TensorFlow, MXNet and TensorRT. Supports inference and training phases. Can use real data if dataset is available. Else, falls back to synthetic data. Supports bare metal and docker environments.

If you think it's useful you can read more on: https://github.com/HewlettPackard/dlcookbook-dlbs