Thanks, Laurence! One of the big reasons that LLMs don't get stuck is that they live in such a high-dimensional space. Imagine you're on some 2D surface: if you're in a well, it's hard to get out because you only have two directions to move—but imagine you had tens of thousands of directions to try to get out. It's much easier!

Hmm... annoying!! I absolutely dropped it in there yesterday. I'll make sure it works this time.

As far as DeepSeek goes, as best I can tell there are indeed some architectural improvements, but the overall idea and structure remain the same and most of the improvements are in the training process. This article from MIT Technology Review is insightful! https://www.technologyreview.com/2025/01/31/1110740/how-deepseek-ripped-up-the-ai-playbook-and-why-everyones-going-to-follow-it/

I don't! If you're referring to the Q&A digests, that's because those are just logs of the live chats from the sessions rather than email or otherwise "official"/formal communications. I think you'll find that the number of people who capitalized the first word of their messages back on AIM in 1997 was also quite small—though to your point, I do imagine that that number is shrinking over time as people have more access to instant back-and-forth communication. Language does change over time, and while I am largely a stickler for rules in a visceral sense I'm certainly not a prescriptivist. (If I were, I might point out that you use a hyphen rather than an em-dash in your last sentence, and add a novel space before the ellipses :). Language—written and spoken—always changes, particularly when exacerbated by the movement to new mediums and entry tools like keyboards where it's easier to type a hyphen than an en- or em-dash.)

Gradient decent optimization procedures can get trapped at falsw (local) minimums. How does Chat GPT avoid or correct false mionimums? Aside Incredible clas, many thanks Arbin.

Added!! Sorry about that!

Arben, On Wednesday I had asked, "Is DeepSeek doing something fundamentally different, or did they find a way to do it more efficiently?" You responded, "I have to say I'm not sure on that front—I'll try to look into this tomorrow and post about it in the Community thread." Have you had a chance to look into this yet?

Also, I am not seeing a Digest_Day5 in the Daily Q&A Digests.

Arben, Thanks 10^6 for a wonderful course! Question: Will you be posting an updated Q&A Digest along with an updated notebook for Day 5? We would appreciate it very much.

In response to the survey question, "How will you use what you learned at this Daily Study Group?" I wrote: I will have a better appreciation and understanding of what is being discussed in the numerous newscasts, podcasts, articles, interviews that one encounters about ChatGPT or other LLMs. Something the course did not address: How could the technology overviewed lead to the catastrophic results people are predicting about AGI? One would not think that even monstrously large matrices could produce malicious consciousness. (I am sometimes reminded of the book I read as a teenager, "Colossus: The Forbin Project", a 1966 science fiction novel by D. F. Jones, about super-computers taking control of mankind.) The things Arben demoed are really impressive, but there is no "mind" (with potential intensions) producing them. I would like to hear Arben's thoughts about these issues. Even more, can you point us somewhere that Stephen Wolfram has discussed these issues (potential threats of AI or AGI)?

Here is a good expert discussion: https://drive.google.com/file/d/1JVPc3ObMP1L2a53T5LA1xxKXM6DAwEiC/view

Also: https://www.google.com/books/edition/The_Age_of_AI/Y2QwEAAAQBAJ?hl=en&gbpv=1

This could be considered a post-DeepSeek update to the reference above: https://www.csis.org/analysis/deepseek-huawei-export-controls-and-future-us-china-ai-race