This is a much appreciated start. I still look forward to being able to train a neural net on my Mac in Wolfram Language using its GPU and hope that will come in 14.2 or a Paclet Update.

This is really very slow and insignificant progress, so far. This discussion thread started 3 years ago and there is still no progress. The Metal package for GPU computing on Apple silicon is really the starting point only. Its recent performance is poor.

I am very sceptic regarding notable change of GPU computing on Apple silicon. NVIDIA-CUDA is the far more advanced solution.

MPS covers a broad spectrum of tasks, but they are defined tasks not generic GPU operations.

Thinking about this there is a Khronos SPIRV to Metal converter that works well for compute shaders, one would have to convert from GLSL or some other language to SPIRV within the confines of the limits of Metal to do this.

I have written a GLSL to Metal converter using this path, its quite simple and wouldn't be too hard to write as a FFI to Mathematica.

Responding to Yaroslav: I think that this is a bad advice. First, even though the headquarters for Wolfram Reserch is, indeed, in Illinois (hardly a backwater), the company itself is about as global as one can get.

Second, for the most part, Wolfram Research has been a leader in new technologies, from support of the GUI in Next and Macintosh, to the development of Wolfram|Alpha.

Third, while Apple sometimes tries out new technology (OpenDoc, Touch bar, etc.) that it later abandons, the use of Metal is now almost 10 years old, and is a matter of active development. Further, it would be in Wolfram's interest to make use of Apple's APIs, since they isolate the code from the hardware.

Fourth (and this relates to #3), NVIDIA may be the leader, but the technology is hard to work with and breaks frequently. (I used the technology when Apple used NVIDIA).

Now, I have something at stake here: all my computers use Apple Silicon, and I would really like to be able to use the option "UseGPU" and have it accelerate my computations. Based on what I have seen on the current implementation in Mathematica, it would certainly be worth the effort.

I am sure that there a lot of people who would like to take full advantage of things like Machine learning and neural nets that would benefit from using the Apple Silicon GPUs, but who would rather not have to learn the low level nuts and bolts, or, perhaps do not have funding to invest in a lot of extra hardware.

Recent comments from Stephen seem to indicate that taking full advantage of the new Apple hardware is under active development at Wolfram, so I am hopeful that some version of Mathematica (maybe even some version of 13.x) will do what we want.

As an advice to Wolfram Research, I would keep track of what other computation frameworks do, and follow their lead. Accelerator surface keeps changing, and being in Illinois, it's probably hard to keep track of the latest trends. API may look good until you meet someone for a beer and find that leadership has pulled the plug on maintenance (cough MxNET cough)

I found a relevant issue on Jax github and they aren't planning to support M1 GPUs yet https://github.com/google/jax/issues/8074

Other places to check are Julia numeric computing community, TensorFlow, someone eventually is going to get seduced by unused FLOPs in their laptop and figure out a good solutuon.

"What I would like to see is that any function in Wolfram language that has an option "UseGPU" will actually use the GPU on my Mac in the not too distant future. I don't care that I could get faster results by investing in different hardware (and spending the time and money to make it work). For almost everything else that I need my computer to do, Mathematica and Wolfram Language give me the tools I need to do a much better job than I could do myself. It is only the lack of Wolfram's use of the GPU and related hardware technologies (Neural engine, etc.) that is an issue."

Exactly.

Stephen noted that he uses a Mac Pro in his February 2019 entry on his blog. At that time, "Mac Pro" referred to the 2013 "trash can" model.

I could see him working quite well with a 16GB M1 Mac Mini with 2TB of storage. You can drive two 4K displays with that machine. It appears that Apple's swapping performance is improved in this version of the OS. Coupled with the faster SD access, that can compensate for the smaller RAM. As a bonus, you can get exactly the same mobile processor/RAM/storage on the M1 Macbook Air.

The thermals of this processor are astonishing. Many of the reviewers noted they went for days without recharging the battery. The MBA is built without a fan, but it takes many minutes of full load before it gets thermally throttled. Anyone who uses the current Intel processors to keep their lap warm will have to get a cat.

I hope the M1 becomes the de facto platform for Wolfram's technical staff. I'd love to see Wolfram's engine take advantage of the M1's GPU and Neural Engine for outstanding performance on the Mac. That same code should work for running code on the iPhone and iPad.

Are you sure? Benchmarks I have seen for the new chips would indicate otherwise. Granted, these numbers are not direct comparisons of neural net computations, but it probably indicates what is possible.

As I see it, the main sticking point is that the open-source software Wolfram uses for GPU acceleration of neural net computations is dominated by NVIDIA. The trick, which I think is possible done by a small group of people, would be to make a function-call equivalent library that uses the Apple technology.

This should be within the capabilities of Wolfram ore even people in this community. At the very least, macOS and iOS (iPadOS) users would be able to do computations significantly faster than they can now. Even if it can't match NVIDIA, which is debatable, it would be orders of magnitude faster than just using the CPU.

Note that the M1 chip does not support eGPUs. It is possible that the chips that Apple ends up putting in the Mac Pro or iMac Pro will support these add-ons. Most of us don't have the deep pockets (or ample grant money) to invest in this solution, though.

I fooled around with this stuff back at the time when Apple used NVIDIA. What was frustrating was the fact that NVIDIA would change which cards were supported, and so you could never depend on the resource being available. This is certainly the case now for people using Windows or Linux Systems. The nice thing about Apple is that their APIs are abstracted from the hardware, so if the hardware changes (which it just did in a big way), the APIs do not.

I can understand the desire on the part of Wolfram to use cross-platform solutions whenever possible. However, Wolfram has, in the past, made use of specific hardware and OS functionality to fully exploit any advantages. Remember that the Notebook paradigm was available on the Mac (and Next) a long time before Windows because versions of Windows before Win 95 were not very capable. In addition, Mathematica users on Macs could make full use of the 32-bit (and 64-bit) architecture while Windows computers were still hobbled by 16 bit CPUs. Now the we have a glimpse of what Apple Silicon can do, I hope that Wolfram Research will once again take advantage of hardware and OS opportunities as it has in the past.

I have not done this type of coding for some time, or I would be tempted to take on the task myself.

I must apologize for getting this thread off on a tangent. Ultimately, the issue is not whether Apple's neural net/GPU engine is better than NVIDIA, but whether Mathematica will make full use of the hardware available to macOS (and probably iPadOS) users.

After what in my opinion is a slow start, Mathematica is making use of Metal (rather than relying on OpenGL), and the results are very good. This has permitted expansion into new areas, such as the very experimental ray tracing in the 12.2 beta.

All I am looking for as a long-time Mathematica on Macintosh user is that the software make full use of the available hardware. I don't care if NVIDIA is faster on different hardware, as long I have decent hardware acceleration with my hardware.

Mathematica has been characterized as a swiss army knife. The analogy is not exact, since software does not place the same constraints as physical design. However, the analogy is apt in the following sense: it meets the needs of the vast majority of users who need to use some mathematical techniques or C/S magic without having to deal with the messy details. I look at the developers of Mathematica 's functions as collaborators in a very real sense. Back in the dark ages, I filled that role when I worked in a research lab, but those says are long gone.

In the case of Machine learning, I have no doubt that there are dedicated tools that doo a better job, just as there are better tools for audio and image processing -- although there are some things that Mathematica can do that dedicated programs do not, simply because of access to mathematical functionality created for another purpose.

I am not concerned with the availability of optimized libraries similar to those for Intel (and NVIDIA). Worst case, the Rosetta II emulator should be able to handle this functionality and still be faster than than running on Intel. However, Apple has the resources to do what you suggest -- and they have already done so with previous transitions. Hardware evolves all the time. It can be painful, but that is what it is, living on the frontier. It is perhaps my perspective of having coded for nearly 50 years, that I have a different context.

Bottom line

What I would like to see is that any function in Wolfram language that has an option "UseGPU" will actually use the GPU on my Mac in the not too distant future. I don't care that I could get faster results by investing in different hardware (and spending the time and money to make it work). For almost everything else that I need my computer to do, Mathematica and Wolfram Language give me the tools I need to do a much better job than I could do myself. It is only the lack of Wolfram's use of the GPU and related hardware technologies (Neural engine, etc.) that is an issue.

I have spoken with several Mathematica users who use macOS, both within Wolfram Research and elsewhere, and this is something that all of them would like.