# WolframMark System Comparison and System Benchmarks

GROUPS:
 Jacob Lapenna 1 Vote I was wondering what criteria is/was used to determine what benchmarks to include when running a benchmark report on a particular machine. Specifically, running: Needs["Benchmarking"] BenchmarkReport[] After running this on my work machine, I noticed the top benchmark was an Intel Core i7-3770 CPU @ 3.40 GHz (8 cores) running 64-bit Linux. Surely there are better (i.e. higher WolframMark "score") architectures out there to compare, right? After briefly searching the web and other forums for other people using the benchmarking package, I noticed the same benchmarks for comparison on other posted reports. Have the CPU benchmarks not been updated to current technology? Perhaps if I had a better architecture I might see my results show up above this benchmark? Has anyone done this?
2 months ago
8 Replies
 Those example measurements in the Benchmarking package are useless. They haven't changed in ages. They are exactly the same even in the oldest version I have installed, which is 10.0 from 2014.If you open up the package source, or evaluate BenchmarkingPrivate\$BenchmarkLibrary you will see that there are some results from 2010 / M8.0 and some form 2014 / M10.0Mathematica's performance has improved a lot between 8.0 and 10.0, and again a lot since Furthermore, did the benchmarks themselves change? We do not know. Thus comparisons are quite meaningless.The benchmarking package is interesting if you compare two computers with the same M version, or compare successive versions of M (and manually verify that the benchmarks have not changed). But even then, it is probably in need of some updates ...
2 months ago
 Szabolcs Horvát 9 Votes If there are enough interested people, it would be more useful to start a community benchmarking project, with open discussion/feedback to ensure quality and versioning to ensure reproducibility.
2 months ago
 Sander Huisman 1 Vote That would indeed be a cool idea! A very broad test-set, but might be tricky to make it compatible with older versions I guess? Or provide a N/A when it can not be ran?
2 months ago
 Karl Unterkofler used to run MMA benchmarking site in the MathGroup days.
2 months ago
 Just to bump up the topic, in a cordial way..., and waiting for eventual proactivity on the centralized and versioning system, here's a benchmark on an AMD Ryzen 7 1700 eight core 3.00 GHz, with Mathematica 11.2.0 (information currently in demand by some curious minds). Attachments:
 For further comparison: System: Mac OS X x86 (64-bit) Date: September 28, 2017 Wolfram Language Version: 11.2.0 Benchmark Result: 2.98 `(When originally posting this, I forgot to start with a fresh kernel. Doing so bumped the benchmark result from 2.40 to 2.98.)