"What I would like to see is that any function in Wolfram language that has an option "UseGPU" will actually use the GPU on my Mac in the not too distant future. I don't care that I could get faster results by investing in different hardware (and spending the time and money to make it work). For almost everything else that I need my computer to do, Mathematica and Wolfram Language give me the tools I need to do a much better job than I could do myself. It is only the lack of Wolfram's use of the GPU and related hardware technologies (Neural engine, etc.) that is an issue."

Exactly.

Are you sure? Benchmarks I have seen for the new chips would indicate otherwise. Granted, these numbers are not direct comparisons of neural net computations, but it probably indicates what is possible.

As I see it, the main sticking point is that the open-source software Wolfram uses for GPU acceleration of neural net computations is dominated by NVIDIA. The trick, which I think is possible done by a small group of people, would be to make a function-call equivalent library that uses the Apple technology.

This should be within the capabilities of Wolfram ore even people in this community. At the very least, macOS and iOS (iPadOS) users would be able to do computations significantly faster than they can now. Even if it can't match NVIDIA, which is debatable, it would be orders of magnitude faster than just using the CPU.

Note that the M1 chip does not support eGPUs. It is possible that the chips that Apple ends up putting in the Mac Pro or iMac Pro will support these add-ons. Most of us don't have the deep pockets (or ample grant money) to invest in this solution, though.

I fooled around with this stuff back at the time when Apple used NVIDIA. What was frustrating was the fact that NVIDIA would change which cards were supported, and so you could never depend on the resource being available. This is certainly the case now for people using Windows or Linux Systems. The nice thing about Apple is that their APIs are abstracted from the hardware, so if the hardware changes (which it just did in a big way), the APIs do not.

I can understand the desire on the part of Wolfram to use cross-platform solutions whenever possible. However, Wolfram has, in the past, made use of specific hardware and OS functionality to fully exploit any advantages. Remember that the Notebook paradigm was available on the Mac (and Next) a long time before Windows because versions of Windows before Win 95 were not very capable. In addition, Mathematica users on Macs could make full use of the 32-bit (and 64-bit) architecture while Windows computers were still hobbled by 16 bit CPUs. Now the we have a glimpse of what Apple Silicon can do, I hope that Wolfram Research will once again take advantage of hardware and OS opportunities as it has in the past.

I have not done this type of coding for some time, or I would be tempted to take on the task myself.

Not a chance in the near future (about few years? ). There is no other technology which is able to be comparable with Nvida CUDA.

Stephen noted that he uses a Mac Pro in his February 2019 entry on his blog. At that time, "Mac Pro" referred to the 2013 "trash can" model.

I could see him working quite well with a 16GB M1 Mac Mini with 2TB of storage. You can drive two 4K displays with that machine. It appears that Apple's swapping performance is improved in this version of the OS. Coupled with the faster SD access, that can compensate for the smaller RAM. As a bonus, you can get exactly the same mobile processor/RAM/storage on the M1 Macbook Air.

The thermals of this processor are astonishing. Many of the reviewers noted they went for days without recharging the battery. The MBA is built without a fan, but it takes many minutes of full load before it gets thermally throttled. Anyone who uses the current Intel processors to keep their lap warm will have to get a cat.

I hope the M1 becomes the de facto platform for Wolfram's technical staff. I'd love to see Wolfram's engine take advantage of the M1's GPU and Neural Engine for outstanding performance on the Mac. That same code should work for running code on the iPhone and iPad.

Responding to Yaroslav: I think that this is a bad advice. First, even though the headquarters for Wolfram Reserch is, indeed, in Illinois (hardly a backwater), the company itself is about as global as one can get.

Second, for the most part, Wolfram Research has been a leader in new technologies, from support of the GUI in Next and Macintosh, to the development of Wolfram|Alpha.

Third, while Apple sometimes tries out new technology (OpenDoc, Touch bar, etc.) that it later abandons, the use of Metal is now almost 10 years old, and is a matter of active development. Further, it would be in Wolfram's interest to make use of Apple's APIs, since they isolate the code from the hardware.

Fourth (and this relates to #3), NVIDIA may be the leader, but the technology is hard to work with and breaks frequently. (I used the technology when Apple used NVIDIA).

Now, I have something at stake here: all my computers use Apple Silicon, and I would really like to be able to use the option "UseGPU" and have it accelerate my computations. Based on what I have seen on the current implementation in Mathematica, it would certainly be worth the effort.

I am sure that there a lot of people who would like to take full advantage of things like Machine learning and neural nets that would benefit from using the Apple Silicon GPUs, but who would rather not have to learn the low level nuts and bolts, or, perhaps do not have funding to invest in a lot of extra hardware.

Recent comments from Stephen seem to indicate that taking full advantage of the new Apple hardware is under active development at Wolfram, so I am hopeful that some version of Mathematica (maybe even some version of 13.x) will do what we want.

It would indeed be nice to have "use GPU" option and have things run fast automatically.

The reason people use Nvidia accelerators is because they invested so much into software layer. Your accelerator may have the flops, but you also need the software to move the correct bits to the correct transistors! In particular, it's libraries like CuDNN/CuTensor.

AMD doesn't really have an equivalent so people either have to make their own version, which is an option for someone like Google, or pay a large markup for Nvidia offering.

Computational scientists went from doing GPGPU by

1. programming shader units, to
2. writing CUDA, to
3. reusing high quality primitives like GEMM.

Incidentally, the reason we have CUDA because of a bet Nvidia made in early 2000s on virtual reality worlds. Jensen pushed to sacrifice some of the rendering performance to enable ability to do physics simulations inside the GPU. This decision almost bankrupted the company at the time, but years later it allowed the pivot from gaming into high-performance computing. You can see this dark period reflected in Nvidia's stock price

DateListPlot@FinancialData["NVDA", {"Jan. 1, 2000", "Jan. 1, 2008"}]

Now, M1 Max is out, gaming performance is great, how do you make use of these 57 billion transistors for general purpose computing?

You could either:

1. Go back to programming shaders or
2. Wait for someone else to program these shaders. This is an active area of development, a good discussion here

MPS covers a broad spectrum of tasks, but they are defined tasks not generic GPU operations.

Thinking about this there is a Khronos SPIRV to Metal converter that works well for compute shaders, one would have to convert from GLSL or some other language to SPIRV within the confines of the limits of Metal to do this.

I have written a GLSL to Metal converter using this path, its quite simple and wouldn't be too hard to write as a FFI to Mathematica.