The Metal package for GPU computing on Aplle silicon is really the starting point only. It's recent performance is very poor. I am very sceptic regarding significant change of GPU computing on Apple silicon. NVIDIA is far more enhanced and advanced.

Hear, hear. Metal is only one alternative for accessing GPUs on the mac. One need only search on

site:developer.apple.com wwdc 2024 "machine learning"

to see the diverse offerings that Apple has for accessing non-CPU on-chip processors. NVIDIA has gotten the jump in the past few years, but Wolfram Research has a large number of staff using Apple silicon computers as their principal platform. Apple has strategically committed to its CPU/GPU//Neural Engine architecture to span everything from iPhones to iPads to laptops to workstations. Apple also announced custom in-house Apple silicon cloud servers for secure private machine learning applications at WWDC 2024. From handheld devices to server farms, Apple is committed to the same homogenous hardware architecture.

Apple and Wolfram Research are a winning combination. I believe that embracing the Apple Silicon platform is crucial to WR's future (and vice versa).

This is a much appreciated start. I still look forward to being able to train a neural net on my Mac in Wolfram Language using its GPU and hope that will come in 14.2 or a Paclet Update.

Check out this project: MetalLink: GPU computations on Apple Silicon

https://community.wolfram.com/groups/-/m/t/3189524

Actually, after looking at MPS a bit, it seems these are not your graphics shaders from 2005, they have higher level primitives built-in

Responding to Yaroslav: I think that this is a bad advice. First, even though the headquarters for Wolfram Reserch is, indeed, in Illinois (hardly a backwater), the company itself is about as global as one can get.

Second, for the most part, Wolfram Research has been a leader in new technologies, from support of the GUI in Next and Macintosh, to the development of Wolfram|Alpha.

Third, while Apple sometimes tries out new technology (OpenDoc, Touch bar, etc.) that it later abandons, the use of Metal is now almost 10 years old, and is a matter of active development. Further, it would be in Wolfram's interest to make use of Apple's APIs, since they isolate the code from the hardware.

Fourth (and this relates to #3), NVIDIA may be the leader, but the technology is hard to work with and breaks frequently. (I used the technology when Apple used NVIDIA).

Now, I have something at stake here: all my computers use Apple Silicon, and I would really like to be able to use the option "UseGPU" and have it accelerate my computations. Based on what I have seen on the current implementation in Mathematica, it would certainly be worth the effort.

I am sure that there a lot of people who would like to take full advantage of things like Machine learning and neural nets that would benefit from using the Apple Silicon GPUs, but who would rather not have to learn the low level nuts and bolts, or, perhaps do not have funding to invest in a lot of extra hardware.

Recent comments from Stephen seem to indicate that taking full advantage of the new Apple hardware is under active development at Wolfram, so I am hopeful that some version of Mathematica (maybe even some version of 13.x) will do what we want.

Would love to see this as well, I'm so amazed at what my mid-spec M1 can do that my daughter is probably getting it soon as i'm going to upgrade to a M1Max. Support for the GPU and/or NeuralEngine would be awesome. Having said that though, for most of the ML stuff I'm doing, the support for remote job execution in AWS is sufficent. I just hate not being able to use all the power on my local machine.

Also, I believe a lot of the ML support is delegated out to MXNet so presumably once it supports Metal, etc, Mathematica's ML funcs will as well. That would be a step towards more general support.

Thanks. I should have led with that.

I hope someone from Wolfram management is looking at this thread.

I must apologize for getting this thread off on a tangent. Ultimately, the issue is not whether Apple's neural net/GPU engine is better than NVIDIA, but whether Mathematica will make full use of the hardware available to macOS (and probably iPadOS) users.

After what in my opinion is a slow start, Mathematica is making use of Metal (rather than relying on OpenGL), and the results are very good. This has permitted expansion into new areas, such as the very experimental ray tracing in the 12.2 beta.

All I am looking for as a long-time Mathematica on Macintosh user is that the software make full use of the available hardware. I don't care if NVIDIA is faster on different hardware, as long I have decent hardware acceleration with my hardware.

Mathematica has been characterized as a swiss army knife. The analogy is not exact, since software does not place the same constraints as physical design. However, the analogy is apt in the following sense: it meets the needs of the vast majority of users who need to use some mathematical techniques or C/S magic without having to deal with the messy details. I look at the developers of Mathematica 's functions as collaborators in a very real sense. Back in the dark ages, I filled that role when I worked in a research lab, but those says are long gone.

In the case of Machine learning, I have no doubt that there are dedicated tools that doo a better job, just as there are better tools for audio and image processing -- although there are some things that Mathematica can do that dedicated programs do not, simply because of access to mathematical functionality created for another purpose.

I am not concerned with the availability of optimized libraries similar to those for Intel (and NVIDIA). Worst case, the Rosetta II emulator should be able to handle this functionality and still be faster than than running on Intel. However, Apple has the resources to do what you suggest -- and they have already done so with previous transitions. Hardware evolves all the time. It can be painful, but that is what it is, living on the frontier. It is perhaps my perspective of having coded for nearly 50 years, that I have a different context.

Bottom line

What I would like to see is that any function in Wolfram language that has an option "UseGPU" will actually use the GPU on my Mac in the not too distant future. I don't care that I could get faster results by investing in different hardware (and spending the time and money to make it work). For almost everything else that I need my computer to do, Mathematica and Wolfram Language give me the tools I need to do a much better job than I could do myself. It is only the lack of Wolfram's use of the GPU and related hardware technologies (Neural engine, etc.) that is an issue.

I have spoken with several Mathematica users who use macOS, both within Wolfram Research and elsewhere, and this is something that all of them would like.

Are you sure? Benchmarks I have seen for the new chips would indicate otherwise. Granted, these numbers are not direct comparisons of neural net computations, but it probably indicates what is possible.

As I see it, the main sticking point is that the open-source software Wolfram uses for GPU acceleration of neural net computations is dominated by NVIDIA. The trick, which I think is possible done by a small group of people, would be to make a function-call equivalent library that uses the Apple technology.

This should be within the capabilities of Wolfram ore even people in this community. At the very least, macOS and iOS (iPadOS) users would be able to do computations significantly faster than they can now. Even if it can't match NVIDIA, which is debatable, it would be orders of magnitude faster than just using the CPU.

Note that the M1 chip does not support eGPUs. It is possible that the chips that Apple ends up putting in the Mac Pro or iMac Pro will support these add-ons. Most of us don't have the deep pockets (or ample grant money) to invest in this solution, though.

I fooled around with this stuff back at the time when Apple used NVIDIA. What was frustrating was the fact that NVIDIA would change which cards were supported, and so you could never depend on the resource being available. This is certainly the case now for people using Windows or Linux Systems. The nice thing about Apple is that their APIs are abstracted from the hardware, so if the hardware changes (which it just did in a big way), the APIs do not.

I can understand the desire on the part of Wolfram to use cross-platform solutions whenever possible. However, Wolfram has, in the past, made use of specific hardware and OS functionality to fully exploit any advantages. Remember that the Notebook paradigm was available on the Mac (and Next) a long time before Windows because versions of Windows before Win 95 were not very capable. In addition, Mathematica users on Macs could make full use of the 32-bit (and 64-bit) architecture while Windows computers were still hobbled by 16 bit CPUs. Now the we have a glimpse of what Apple Silicon can do, I hope that Wolfram Research will once again take advantage of hardware and OS opportunities as it has in the past.

I have not done this type of coding for some time, or I would be tempted to take on the task myself.