Hi, Carl. Here's what Grok suggested. "Flush caches" was the only thing I saw that you probably haven't already tried. The response is repeated because I prompted it to "think harder". HTH. https://x.com/i/grok?conversation=1990170094241214480

I have experienced the same thing on two different computers. I think the service is down and has been for several days.

Now that Wolfram is now a serious "services" company, they need status pages that are updated 24/7 with the status of all key cloud services and repositories.

I've wasted several hours on this, including filing a report with Wolfram.

As far as I know, all of the available information we can provide is produced if you give All as a second argument to basically any LLM* function. If it isn't there, I think we must not have it accessible.

Minutes after writing the above post, OpenAI released the GPT-5.1 model series APIs.

For Mathematica (or any coding application), the premier OpenAI model is unquestionably GPT-5.1 Codex. It's designed for exceptional instruction following and code generation. While the full model is intended for high-end planning, architecture, design, refactoring, etc, the Mini model excels at coding.

And the Mini version is cheap. Using the above formula, the price per million tokens is (3 * 0.25 + 1 * 2.00) / 4 or $0.6875.

Unfortunately, while it can now be selected in Mathematica Preferences / AI Settings and used with the LLM APIs (Model name "gpt-5.1-codex-mini"), it will immediately fail (see attached image).

It seems Wolfram is still using the now-deprecated OpenAI "completions" API rather that the modern "responses" API.

Crap!

Attachments:

GPT5.1 API Fail.png

Thank you, Arben!

I would also like to point out that this does not just apply to us poor home users.

Any research project which involves repetitive operations on data (imagine sentiment analysis on a million samples) would have its budget blown out of the water by a more expensive model.

Once one has a viable prototype solution, try it with a Nano model. These are dirt cheap in comparison to even the Minis.

GPT-4.1 Nano is presently the one to test against. Once Wolfram has fixed their GPT-5 and GPT-5.1 interface code, these Nano versions are even cheaper.

Arben... Thanks for this. Really helpful.

Something I wish to point out. Perhaps this should be a support request but, for LLM stuff, they never know what I'm talking about.

When I do a fresh start of Mathematics (new 14.3 download and clean install with packlet update on Windows 11) and then execute your code, I get the error shown in "getServiceModelList.png". It then gives me the LLM model association. This will only happen once. I have to restart Mathematica or quit the kernel to repeat the error.

The same error occurs the first time I open Preferences / AI Settings in a kernel session, as shown in "AI Settings.png".

Attachments:

AI Settings.png

getServiceModelList.png

It looks like "Reasoning" is an available setting for LLMEvaluator, but there seems to be a bug in it right now. I believe that it's fixed in our next major release and they're working on backporting it into a paclet update that will work in earlier versions. (And once 5.1 is working, that defaults to setting "Reasoning" to "None" anyway, which is not an option on the gpt-5 models [which default to "Medium"].)

This is my current understanding, at least, but I can't claim to be fully authoritative on this.

"https://www.perplexity.ai/search/tell-me-a-funny-joke-no-one-ha-cKpP47u6TNageZo465bXCQ#0"

One-second response with GPT-5.1. I tried it multiple times.

This on the first day of the 5.1 API release - and it will only get better.

The webpage I was asking about today was the Wolfram Support Document #62525, How can I start using Chat Notebooks in Wolfram products? It was announced in the chat panel on Monday by (I believe) Arben. I located it in my browser history. I thought that "Grok" was misspelled on that webpage; I hadn't realized that "Groq" was actually a completely different AI than xAI's product. AIs are easy, but product naming is complicated. :)

The Wolfram LLM Benchmarking Project is a fascinating little project. As one of those many who went through the EIWL test framework, I viewed it as an idiosyncratic set of questions. I'm fascinated the best score from an AI was 64.7% -- especially when the quiz answers are available through the pages of the published books of EIWL. The test framework is doing some sophisticated pattern-matching, and sometimes its choices were completely arbitrary. I (imperfectly) recall one of the exercise's "correct" answers had to do with the names one chose for variables. I'd have to plow through that mass of exercises to find the specifics. If someone internally at Wolfram thought this was a good way to systematically benchmark all AIs, I'd love to hear a detailed explanation why.

The generated Shakespearian AI output on Monday reminded me of one of the original "computational essays". The DeCSS Haiku was created in 2001 as a means of protesting the arrest of another programmer for creating the original DeCSS software. The code would allow decryption of a DVD: something Hollywood tried to prohibit in the late 1990s. Seth Schoen created the 465 stanza haiku to create "art" that was precise computational instructions for re-implementing the DeCSS. In this blog post, he says it took him about 13 hours to make almost all of the haiku. AIs could probably repeat his effort in a few minutes. AIs could also generate code in the language of choice to re-generate a DeCSS implementation from that original 465-stanza haiku. OTOH, current AIs are a terrible choice for either project.

Hey Arben, As always, when you teach a class, I sign up. I thought I had an example here but it was premature. But sometimes the notebook assistant get's lost trying to invent functions that don't exist. (when I am being lazy and trying to get the assistant to write some code for me) It seems like an inspiration for writing a function that does what it thinks should exist, but what do you do generally when that happens?

Will you include a demo running a local LLM during this course? Is there anything special about accessing an LLM locally, or does that look the same as accessing remotely?

I believe that 2026 will be the first year that local LLM usage becomes widespread. With their A19 Pro and M5 processors, Apple has straightened out a major kink in their architecture. When Apple announced the iPhone 17 models at their September 9 press event, they included a brief discussion of the A19 Pro's new GPU accelerator in the event:

We have been at the forefront of AI acceleration since we first introduced the neural engine eight years ago. We later brought machine-learning accelerators to our CPUs. And while our GPU has always been great at AI compute, we’re now taking a big step forward, building neural accelerators into each GPU core, delivering up to three times the peak GPU compute of A18 Pro. This is MacBook Pro levels of compute in an iPhone, perfect for GPU-intensive AI workloads.

That was a bit vague. Fortunately, the "Petar Tech" YT channel explained more in the video Demystifying Apple's AMX AI accelerator: when and why it outperforms the GPU. In a nutshell, matrix multiplication -- a cornerstone of LLM training and token generation -- happens atomically in this chip. Historically, the elements of the two matrices being multiplied are fetched from memory multiple times. Petar notes in his video commentary: Because nowadays, the energy needed to transfer data significantly exceeds the energy needed to perform computations on the data. Apparently, NVIDIA has been doing this for several generations on their Tensor Cores; Apple is just catching up on this particular processor feature.

Apple has incorporated the same per-GPU accelerators in its M5 Processors that were released for the low-end MacBook Pro and iPad Pro models this month. In one early test, "Time to First Token" was measured about 3.6x faster than a M4 processor. That baseline M5 processor is limited to 32GB of unified memory, but the "M5 Pro" and "M5 Max" chip computers (both available early 2026) should have up to 128GB of unified memory available. Now that Apple has had time to percolate this change through the years of chip layout and fab, we should be able to run some general-purpose LLMs locally starting soon. All of Apple's M5 processor computers should be far more attractive than earlier processors for any sort of AI development or usage.

Hi Quan—could you post a screenshot or copy and paste the output here? I can talk to our team and/or submit a bug if I have a bit more information. Thanks!

I saw an article today on Apple allowing linking of multiple Mac Studios via Thunderbolt 5 in the next version of macOS.

https://www.engadget.com/ai/you-can-turn-a-cluster-of-macs-into-an-ai-supercomputer-in-macos-tahoe-262-191500778.html

As with most any tool, I think there can be good uses for it. Materially speaking, however, I do not have a great view of how things are playing out more broadly. To keep it short and not-so-sweet: (American) society has evolved such that you need a degree to get most any kind of job (if you can get one in the first place). Many universities have taken this situation to heart, and we end up in a situation where said universities function less as institutions that help to create "whole", well-rounded people and instead simply meet the demand for diploma generation. This creates situations where students take classes that they don't want to take being taught by instructors who don't want to teach them. An insidious thing about LLMs is that they enable this process—I feel like maybe Zizek said something about this years ago—but it's very possible that we end up in a situation where the students in classes they don't want to take submit papers that they didn't actually write and those papers are not actually graded by the instructors who don't want to be there either, you know?

I think LLMs can be handy for things like rapid retrieval from dense texts and for as introductory tools or augmentations for tasks like coding, but—as I mentioned in these last two study groups!—fundamentally LLMs do not have access to truth; they do not have an internal sense of the world that allows them to know things in the traditional sense. (Say: you or I know that 1+1=2; an LLM merely finds it exceedingly probable. There is a difference.) In an increasingly post-truth world, I don't think it's too hard to see how this can be a problem on several important axes.

I don't think I can say much beyond this, and I must stress that even this is solely my personal view which I provide for a familiar face here :).

Thanks for letting me know—I'll get this filed with the appropriate team, in case they're not already aware.

Actually, Paul, could you run this and report back? PacletInstall["Wolfram/Chatbook"] We can't reproduce it in 14.3, or pre-release versions, so it might be that your paclet is out of date and hasn't auto-updated yet.

I fixed this problem

In Day 1 of the class (Monday), the use of the forward slash key within a chat is discussed. It's purpose is to enter Wolfram Language in the chat. On my Mac, running 15.7.2 and Wolfram 14.3, When I push the forward slash key (above return), it is not entered. And so nothing happens. Any ideas? I have TextExpander turned off. I can enter forward slash in other applications.

So it does work in a new notebook. Didn't even restart Wolfram Mathematica. But wont' work in exiting notebook that I just created yesterday. Very strange.

Ah, yes! Many interactive elements in notebooks subtly depend on dynamic updating's being enabled.