@Arben , I attended the course live on Friday and am going through the other sessions through the recordings. Good job with the poll questions. They worked great in the BigMarker replays. I was able to understand the poll questions as you asked them. Also, you consistently shared the results on your screen; people listening that way could play along. It's a good quiz, too.

I loved your discussion about particles in the universe on Day 1. It tickled me immensely to see an instructor provide a calculation that uses [Wolfram's estimate of] the number of particles in the universe.

That discussion reminded me of a short story by Stanislaw Lem in the book The Cyberiad. In the collection, a pair of "constructor" robots create fantastic machines. In the sixth sally, they create a demon of the second kind to work their way out of a problem. The story references a demon of the first kind -- Maxwell's Demon. Maxwell's demon was about thermodynamics; this second demon was about information. For a book that was created over 50 years ago, Lem's machine in this story is a rather astonishing vision of an LLM. Their demon was using a tiny bit of stale air -- a tiny number of particles -- as the source of its information; it could not possibly work as described in the story. OTOH, Maxwell's demon couldn't possibly work, either. Both hypotheticals are quite interesting; they are fine little thought experiments. I don't know if you -- or anyone -- will learn anything from Lem's short story, but you should be amused. Ask an AI to find you a copy.

Hmm... annoying!! I absolutely dropped it in there yesterday. I'll make sure it works this time.

As far as DeepSeek goes, as best I can tell there are indeed some architectural improvements, but the overall idea and structure remain the same and most of the improvements are in the training process. This article from MIT Technology Review is insightful! https://www.technologyreview.com/2025/01/31/1110740/how-deepseek-ripped-up-the-ai-playbook-and-why-everyones-going-to-follow-it/

Added!! Sorry about that!

Arben, Thanks 10^6 for a wonderful course! Question: Will you be posting an updated Q&A Digest along with an updated notebook for Day 5? We would appreciate it very much.

I was wondering in the last class two topics related to line of research in Wolfram:

1) Is Wolfram Research developing their own Algorithmic Differentiation framework for programs, a.k.a (parametrized)functions, that depends on loops, conditionals, recursion, and (non)smooth elemental functions? I commented in class this is between numerical and symbolic differentiation. By the way, this is the standard in Scientific Machine Learning and the backbone to make computationally efficient optimizers like ADAM (and their subsequent versions).

2) In case that (1) is negative, are you more focused on developing the discrete techniques to emulate neural networks capacity via Rule Arrays applied to cellular automata?

Gradient decent optimization procedures can get trapped at falsw (local) minimums. How does Chat GPT avoid or correct false mionimums? Aside Incredible clas, many thanks Arbin.