Are there any benchmarks yet for Mathematica 12.1 on the new (13") M1 MacBook Air or (13") M1 MacBook Pro?
That would be interesting for me as well. I assume that right now, Mathematica has to use the Rosetta 2 Layer to run on non-x64-architecture?
An easy "benchmark" would be to have them install Mathematica in an Apple Store, run some pre-defined code, and measure the time it needs to perform that task.
After some cursory usage of Mathematica 12.1.1, I can report that it seems to run well on my new MacBook Air M1. Here's a screenshot of the WolframMark Results report:
Given the fact that Mathematica is running under x86 emulation, I'm. pretty impressed with the results.
Thank you for that benchmark!
Which version of the new M1 MacBook Air is this: which processor (8-core CPU and 7-core GPU or 8-core CPU and 8-core GPU, in case Mathematica takes advantage of the GPU, too)? How much "unified memory" (8GB or 16GB, in case that matters?
For a sobering comparison, for my 2019 27-inch iMac with 3.6 GHz 80Core i9 (64GB 2667 MHz DDR4 RAM) running Mathematica 12.1.1 under macOS Catalina 10.15.7, the WolframMark is 4.48. (If I buy a of new MacBook M1 Air or Pro, it will be secondary to the iMac, but I'll still want to run Mathematica on it at a tolerable speed.)
I'd like to see somebody's WolframMark on the new M1 MacBook Pro, too,
Murray, the aforementioned WolframMark benchmark test was ran on an M1 MacBook Air with the 8-core GPU and 16GB of unified memory.
very promising results - could you give the benchmark another try while the MacBook is hot? E.g after watching a HD Youtube Video for some minutes?
As you are surely aware, the CPU gets throttled when too hot; maximum speed is only available for a short time, until the aluminum body heats up.
Therefore, the high benchmark scoring might be a little misleading: Surely true when the Laptop is idle and the computation is short. But for longer tasks, Classify or Predict of large Datasets, the CPU might run into the temperature-throttle. Such an issue would not be detected by the Benchmark test which is only a few seconds in duration.
Could you "heat up" the MacBook and run the benchmark again?
Or maybe run the Benchmark 100 times, and give the result of the last run?
Presumably such throttling does not occur with the M1 MacBook Pro, which has a fan rather than just the Air's passive cooling.
Oliver, I did as you suggested and ran the Benchmark function 100 times. The last "BenchmarkResult" reports to be 3.051. While the function was on repeat, the bottom of the MacBook Air became noticeably more warm and the cumulative CPU utilization held steady at around 73%.
How many cores does the Benchmark function use?
The M1 has only four high-performance cores, while most desktop/laptop CPU's have more; thus, to maintain comparability with benchmark results from other CPU's, you'd want the Benchmark function to be limited to four as well (and you'd also want it to be using the high-performance cores only).
Alternately, one could do comparative testing by timing individual single-threaded commands.
From what I observed, the Benchmark function only uses one Kernel. However, when I ran the repeating Benchmark test, I'm fairly certain the OS was spreading the computation over some number of the M1's cores. I don't think there's a way to specify which processor cores should perform specified operations.
That said, I don't agree that the comparison is invalid. If the M1 is faster when using four performance cores and four power efficient cores over, say, a 8-core Intel i7, isn't that a valid comparison even though some of the M1's cores are more power efficient?
Yes, the M1 is significantly faster than the i7-3770 listed in the Mathematica Benchmark report, but that processor was released in 2012.
Thus I wasn't considering that score, but rather what Murray Eisenberg obtained with the 8-core i9-9900K in his 2019 iMac. And based on that, the M1's comparative performance for MMA is somewhat slower than expected (even accounting for Rosetta 2), relative to its comparative performance thus far on other benchmarks.
Specifically, the score Murray obtained with the i9-9900K is 50% higher than what you obtained with an M1 Air. By contrast, some (though not all) benchmarks comparing the single-core performance of the M1 to that of the (even faster) i9-10900K have the M1 as faster when the benchmark is run natively, and approximately comparable when run under Rosetta 2. Thus it's natural to want to ensure that this discrepancy isn't because Murray's i9 was able to use all 8 cores.
More broadly, my own interest/curiosity is in assessing the performance of this new technology from Apple and, to do this, I think it's most meaningful to focus on per-core performance, since the current 4-performance-core limitation is merely a characteristic of these first models. Apple will be likely be offering higher-end systems with more cores in 2021.
It was interesting to see the benchmark you posted—thanks for doing that. But, to get a meaningful assessment of MMA's per-core performance on AS, we'll probably need to wait until WRI produces a build that runs natively on AS.
Gotcha. I see what you're saying. It'll be interesting to observe how the M1 performs once Mathematica has been updated to run natively on Apple Silicon. Anecdotally speaking, the non-native Microsoft Office apps seem to run slower on my M1 MacBook Air than they do on my 2012 12-core Mac Pro.
These CPUs which you compared with are too old, I don't believe that.
And My mark is 8.0.
Hoping to refresh this thread.
Has anyone benchmarked the just-released 12.3 on an ARM M1 chip?
I just did, but didn't bother to run multiple times + I have been heavily multitasking.
I have a new M1 MacBook Air. Running the last nightly build for the Apple Silicon version, I got a benchmark of 3.21. not bad for a computer intended for light duty.
You are Apple users, anybody tried to run Mathematica on Windows 10 on ARM64 with their x64 emulator? Does it start or install at all? You need Insider´s Build for that as of June 2021.
Here are some results running Mathematica 12.3.1 natively on the M1 MacBook Air (https://mathematica.stackexchange.com/questions/234881/benchmarking-with-mathematica-v-12-for-up-to-date-comparison-across-different-ma/250736#250736) and M1 MacBook Pro (https://mathematica.stackexchange.com/questions/234950/benchmarks-for-new-m1-macbooks/250779#250779).
Both benchmark at 3.2. That's pretty far off the 4.48 Murray reported for his core i9 iMac (though with a slightly different version—12.1.1; I don't know how much diff. that makes). Is it possible Mathematica's code isn't yet well-optimized for Apple Silicon? As I understand it, Mathematica's performance on Intel benefits from MKL. Perhaps there's not yet an equivalent library available for Apple Silicon.
Of course, ideally you'd want to assess the M1's performance with calculations representative of your own workload, rather than relying on a synthetic benchmark.
I have just made Surface Pro X with Windows 11 (build 10.0.22000.160) to run Mathematica 12.3.1, obviously in emulation mode. I tried BenchmarkReport and this machine got only 0.45, which is very low compared to 3.21 of M1 Mac running Mathematica 12.3.0 which is also emulation, not the native 12.3.1 one for Mac.
I wonder how Mathematica on M1 Mac shows the Benchmark when moving from 12.3.0 to 12.3.1 (i.e. native ARM64).