I totally agree on the way that the cost of using LLMs is hidden. Even though I could easily afford it (and many cannot), I am not willing to pony up real money for technology that is of absolutely no use for me -- other than as a fancy code assistant.

I also agree that there is a lot more chrome in the UI for Mathematica. Some of the functionality is useful, of course, and I am happy to have a lot of the improvements made over time. And, I guess one person's chrome is another person's essential service.

So far, it seems that most of the fancy UI stuff can be turned off, so that is good. I would rather see some of the bugs getting fixed, and some functionality that would be generally useful, such as split windows, implemented. Given limited developer resources, in my opinion, there are better things to do than replacing monospaced code text with something else.

Viewing the video today.

It's great that Theo Gray is back doing development for Wolfram Research! He's clearly a natural for enhancing the Wolfram Notebook implementation with an LLM interface.

After watching the video, the biggest unanswered question: can the LLM be run on my local PC? Do Apple Silicon computers have sufficient GPU and Neural Engine computational resources to run the AI? How much pattern data is required to have the AI do its stuff? Running the LLM in a cloud computer is attractive, but running it standalone on your PC is attractive for completely different reasons.

I have been wondering the same thing myself. As the LLMs are currently constituted, they are almost certainly well beyond the means of even a maxed out 'home' computer.

However, I can't help wondering if it would be possible to create a LLM using Wolfram language that could run on a laptop. My current MacBook Pro has 64 Gbytes RAM, and 2 TByte storage, but current iterations can have more of both, to say nothing of the Mac Studio. I first ran Mathematica (version 1.1) on an SE/30 that had 8 Mbytes RAM (the minimum was 2.5), and that extra RAM almost doubled the cost of the computer -- but it was well worth it. In my opinion, current hardware capacity is being underutilized by Mathematica.

Creating a LLM would be beyond what could be done without linked servers, but running one should be possible. (?)

A useful first step would be to make the current neural net functionality use Apple's hardware, instead of requiring a NVIDIA GPU. Next, applying some of the multi-computational ideas from Stephen's physics project might be better than the current effort, which seems to be more 'brute force' than necessary.

I would not mind if the LLM 'specialized' in Wolfram Language code. Such specialization might require fewer resources, and would avoid a lot of issues with current LLMs. (In tests I've done, the LLM seems to make stuff up at least 50% of the time, which renders it useless for many use cases.)