I am using Mathematica 14.2 and this issue with GPT-5 still persists. I have tried your fix of running the PacletInstall, then it worked for only one time. The second time I try again it still gives this same error "presence_penalty". I checked and it only occurs with GPT-5, all other GPT models run fine.

"Neural accelerators" -- optimally efficient matrix multiplication -- have just been added to the GPUs in Apple's M5 processors. The M5 Pro, M5 Max, and M5 Ultra chips should be mightily impressive for high-octane AI researchers and users. At least machines with M5 Pro and M5 Max should be available in the first part of 2026. It sounds like Apple will suggest multiple Mac Studio boxes with a Thunderbolt 5 "bus" for its highest-end customers to distribute computational tasks to multiple machines. The Wolfram Language core numeric computation engine could be seriously enhanced over the next software update or two.

A question for @Arben: it seems as if the Wolfram AI Course Assistant would be ideal for any questions in a Wolfram course -- including double-checking answers to Wolfram quiz questions. As a member of the faculty, are you encouraged or discouraged by the way students could use these marvelous tools?

Thanks Paul,

I agree. I too tried it on a different computer without any success. It would be nice to at least show a message that says "Hey, it's us, don't waste time, we will be back as soon as we can"

I am sure an LLM can come up with something to that effect..

Carl

So you are not experiencing the same issue?

Yesterday and today when I try to access my LLM kit, whether by opening a new notebook or using ' in a notebook, I keep getting this message:

[WarningSign] Cannot check LLMKit subscription status. Contact us » [SpanFromLeft]

Most of my "play time' in Wolfram is over the weekend, and it does not seem there is anybody at Wolfram providing technical support on Saturdays and Sundays....

Does anybody know what this is? Is it on my end or Wolfram's? I have a valid subscription and it was working on Thursday.

"https://www.perplexity.ai/search/tell-me-a-funny-joke-no-one-ha-cKpP47u6TNageZo465bXCQ#0"

One-second response with GPT-5.1. I tried it multiple times.

This on the first day of the 5.1 API release - and it will only get better.

It looks like "Reasoning" is an available setting for LLMEvaluator, but there seems to be a bug in it right now. I believe that it's fixed in our next major release and they're working on backporting it into a paclet update that will work in earlier versions. (And once 5.1 is working, that defaults to setting "Reasoning" to "None" anyway, which is not an option on the gpt-5 models [which default to "Medium"].)

This is my current understanding, at least, but I can't claim to be fully authoritative on this.

Thank you, Arben!

I would also like to point out that this does not just apply to us poor home users.

Any research project which involves repetitive operations on data (imagine sentiment analysis on a million samples) would have its budget blown out of the water by a more expensive model.

Once one has a viable prototype solution, try it with a Nano model. These are dirt cheap in comparison to even the Minis.

GPT-4.1 Nano is presently the one to test against. Once Wolfram has fixed their GPT-5 and GPT-5.1 interface code, these Nano versions are even cheaper.

OK, last LLM question of the evening. I promise!

Arben... Is there a way to hook into the (actual raw text) request-response conversation between Mathematica and the LLM service provider? Or, is there a way to get a log of this?

Arben... Thanks for this. Really helpful.

Something I wish to point out. Perhaps this should be a support request but, for LLM stuff, they never know what I'm talking about.

When I do a fresh start of Mathematics (new 14.3 download and clean install with packlet update on Windows 11) and then execute your code, I get the error shown in "getServiceModelList.png". It then gives me the LLM model association. This will only happen once. I have to restart Mathematica or quit the kernel to repeat the error.

The same error occurs the first time I open Preferences / AI Settings in a kernel session, as shown in "AI Settings.png".

Attachments:

AI Settings.png

getServiceModelList.png

Re: getting a list of models, here's how you can do that:

In[139]:= services = Keys[(LLMServices`LLMServiceInformation[])[[1]]]

You can then grab the models associated with a given service with:

In[142]:= Wolfram`Chatbook`Common`getServiceModelList["OpenAI"]

And just the model names explicitly with:

#Name & /@ Wolfram`Chatbook`Common`getServiceModelList["OpenAI"]

For anybody who was interested in extracting the tokens for a given input from GPT2 (or any other net model that tokenizes that you can get into a notebook, you just need to do this):

tokenizer = 
 NetExtract[NetModel["GPT2 Transformer Trained on WebText Data"], 
  "Input"]

Then, you can run tokenizer["whatever text here"] and it will produce the tokens. For example:

In[591]:= tokenizer["Hello there! My name is Arben. Hello hello hello hello hello goodbye goodbye."]

Out[591]= {15241, 357, 50257, 1756, 1183, 63, 688, 11467, 50244, 18180, 23493, 23493, 23493, 23493, 24574, 24574, 50244}

The webpage I was asking about today was the Wolfram Support Document #62525, How can I start using Chat Notebooks in Wolfram products? It was announced in the chat panel on Monday by (I believe) Arben. I located it in my browser history. I thought that "Grok" was misspelled on that webpage; I hadn't realized that "Groq" was actually a completely different AI than xAI's product. AIs are easy, but product naming is complicated. :)

The Wolfram LLM Benchmarking Project is a fascinating little project. As one of those many who went through the EIWL test framework, I viewed it as an idiosyncratic set of questions. I'm fascinated the best score from an AI was 64.7% -- especially when the quiz answers are available through the pages of the published books of EIWL. The test framework is doing some sophisticated pattern-matching, and sometimes its choices were completely arbitrary. I (imperfectly) recall one of the exercise's "correct" answers had to do with the names one chose for variables. I'd have to plow through that mass of exercises to find the specifics. If someone internally at Wolfram thought this was a good way to systematically benchmark all AIs, I'd love to hear a detailed explanation why.

The generated Shakespearian AI output on Monday reminded me of one of the original "computational essays". The DeCSS Haiku was created in 2001 as a means of protesting the arrest of another programmer for creating the original DeCSS software. The code would allow decryption of a DVD: something Hollywood tried to prohibit in the late 1990s. Seth Schoen created the 465 stanza haiku to create "art" that was precise computational instructions for re-implementing the DeCSS. In this blog post, he says it took him about 13 hours to make almost all of the haiku. AIs could probably repeat his effort in a few minutes. AIs could also generate code in the language of choice to re-generate a DeCSS implementation from that original 465-stanza haiku. OTOH, current AIs are a terrible choice for either project.

Hey Phil! I also watch WWDC ;). I won't be running anything locally for this series, but maybe in the future. (Though I did just get a new laptop from our IT department earlier this year and won't be due for an upgrade for a while, so I won't be benefitting from the M5 improvements! But I really can't complain.)

As far as whether it's any different to run things locally, functionally it isn't. It's more secure, obviously, and relatedly it's more private, but the actual functioning of the thing is not really particularly different. The time it takes to send and receive information to/from LLMs is really not that big a component in the total runtime because it's generally all just text. The extra time taken by network communication may in fact be canceled out by the fact that large companies are already running their setups on specialized hardware, but even if this isn't quite correct it's probably close enough as makes no difference. (It's possible that there are delays based on queuing of their massive numbers of users, though!)

Hey Stephen—could you run the following: ` PacletInstall["Wolfram/Chatbook", UpdatePacletSites -> True] and fully quit and reopen the app and see if it starts to work? It looks like the bug you're reporting was known about in an older version of the Chatbook paclet that powers all of the Chat cell functionality but was fixed a little while back.

(I'm actually going to just recommend everybody run this at tomorrow's session to head off any issues!)

Will you include a demo running a local LLM during this course? Is there anything special about accessing an LLM locally, or does that look the same as accessing remotely?

I believe that 2026 will be the first year that local LLM usage becomes widespread. With their A19 Pro and M5 processors, Apple has straightened out a major kink in their architecture. When Apple announced the iPhone 17 models at their September 9 press event, they included a brief discussion of the A19 Pro's new GPU accelerator in the event:

We have been at the forefront of AI acceleration since we first introduced the neural engine eight years ago. We later brought machine-learning accelerators to our CPUs. And while our GPU has always been great at AI compute, we’re now taking a big step forward, building neural accelerators into each GPU core, delivering up to three times the peak GPU compute of A18 Pro. This is MacBook Pro levels of compute in an iPhone, perfect for GPU-intensive AI workloads.

That was a bit vague. Fortunately, the "Petar Tech" YT channel explained more in the video Demystifying Apple's AMX AI accelerator: when and why it outperforms the GPU. In a nutshell, matrix multiplication -- a cornerstone of LLM training and token generation -- happens atomically in this chip. Historically, the elements of the two matrices being multiplied are fetched from memory multiple times. Petar notes in his video commentary: Because nowadays, the energy needed to transfer data significantly exceeds the energy needed to perform computations on the data. Apparently, NVIDIA has been doing this for several generations on their Tensor Cores; Apple is just catching up on this particular processor feature.

Apple has incorporated the same per-GPU accelerators in its M5 Processors that were released for the low-end MacBook Pro and iPad Pro models this month. In one early test, "Time to First Token" was measured about 3.6x faster than a M4 processor. That baseline M5 processor is limited to 32GB of unified memory, but the "M5 Pro" and "M5 Max" chip computers (both available early 2026) should have up to 128GB of unified memory available. Now that Apple has had time to percolate this change through the years of chip layout and fab, we should be able to run some general-purpose LLMs locally starting soon. All of Apple's M5 processor computers should be far more attractive than earlier processors for any sort of AI development or usage.

This is being looked into :)

Phew, glad to hear it!

Dark mode and chat cells and forward slash do have some problems with readability

So it does work in a new notebook. Didn't even restart Wolfram Mathematica. But wont' work in exiting notebook that I just created yesterday. Very strange.

Hi Arben. My notebooks seem to be stuck going to ChatGPT. How do I undo the? Do I need to get the notebook advisor?

In Day 1 of the class (Monday), the use of the forward slash key within a chat is discussed. It's purpose is to enter Wolfram Language in the chat. On my Mac, running 15.7.2 and Wolfram 14.3, When I push the forward slash key (above return), it is not entered. And so nothing happens. Any ideas? I have TextExpander turned off. I can enter forward slash in other applications.