Several observations:

I absolutely love computational essays as a means to host lectures. They should be used everywhere–for all sorts of online presentations. I loved the example here on Lecture #4 of projecting the "shadow" of a multidimensional array onto a 2D surface. That provides a perfect model for what happens when we make puns: there is some orientation of projection of a multi-dimensional word-space where the "shadow" has particular words line up next to each other. There are many different kinds of puns–just like there are many orientations that one could project the shadows of a word-map.

Indirectly, this course has some profound commentary about learning. In the context of training a neural net, errors are never a problem. Errors are simply a way to re-jigger the neural network to behave better in the future -- to train to a local minimum. Maybe the most important strategy for learning is seeking out strategies-for-possibly-making-errors, noticing when you make errors, and letting the neural network rewire itself as appropriate. We're so obsessed with avoiding [public] errors; that's really silly.

The commentary about what problems AIs cannot solve were as valuable as anything else in this course. I like how Stephen went back to his computational automata to explain this.

Word puzzles and number-puzzles are interesting. When playing knotwords, it's useful to know vowel-consonant combinations and groups of 3 letters. At the same time, that's insufficient to do well at the game. You come face-to-face with strange rules: "if I have guessed a 8-letter word, it must be right". I've gotten better over time, but I couldn't really say how.

Wolfram Research has a compelling value proposition with LLMs + the Wolfram Alpha + the Wolfram Language. It's uncanny how much Wolfram Research's past investments in a vast library of human knowledge are the perfect partner for LLMs. Understanding the shortcomings of LLMs and why they are contained with judicious use of the Wolfram Assistant is an important thing to explain to others.

The Q&A that Arben published from this course are tremendous. Arben's comment that code functions more like a natural language than an algorithm are intriguing. I am paraphrasing his comment; I'll have to scrutinize his written answer.

Hi Gerald—I've taken some time to respond to this one because I'm not sure there's much I can say in my position here. I have personal opinions on this, but we'd kind of get into the weeds more than I think we ought to on Community. I have two main things that I think I can say:

1. LLMs really are quite good at a lot of things, as we've seen. However, some companies suggesting that they're SO powerful that we need to be extremely worried about them "taking over" (et c.) needs to be understood in its proper context—namely, ask "who benefits?" I think that it's highly unlikely that most western governments outright ban or even strongly regulate this sort of stuff, so when a company says this kind of thing, I think the only reasonably-expectable outcome is that they create a public image of their product that they are trying to sell as being unbelievably powerful.

2. At the very least, I think that in the short and medium term the larger danger to society posed by LLMs is the potential for actual, sourced knowledge to be "evaporated". LLMs can learn "the gist" of facts about the world, but they don't know them and generally don't retain them word for word or have the ability to reliably produce sources. As more and more people rely on "what AI said" in terms of forming their opinions, beliefs, and views about certain facts of the world, I think that we run the risk of some bad outcomes. (An unexpected one I just saw recently—somebody who must have been rather young asked for help from other people on a forum, but their request was formatting exactly like a query to an LLM: "Explain it this and this way, and DON'T do this or I won't believe you." Scary!) You have to take this with a grain of salt because I'm an employee here at Wolfram telling you this, but this very issue is why computational language and LLM integration with tools like Wolfram Language is really important: it provides a way for users to be sure that what they're getting from the LLM is related in a concrete way to the real world.

I don't think that there's a 1:1 discussion from Stephen on this topic, but you're likely to find some posts of interest here: https://writings.stephenwolfram.com/category/artificial-intelligence/

I was wondering in the last class two topics related to line of research in Wolfram:

1) Is Wolfram Research developing their own Algorithmic Differentiation framework for programs, a.k.a (parametrized)functions, that depends on loops, conditionals, recursion, and (non)smooth elemental functions? I commented in class this is between numerical and symbolic differentiation. By the way, this is the standard in Scientific Machine Learning and the backbone to make computationally efficient optimizers like ADAM (and their subsequent versions).

2) In case that (1) is negative, are you more focused on developing the discrete techniques to emulate neural networks capacity via Rule Arrays applied to cellular automata?

Also, the idea of converging to a global minimiser of the loss function would not be ideal. We do not want our model to overfit the outputs of the training set, as in the supervised learning scenario.

However, the "descent" direction that an optimiser computes should not be only descent. Why? Because the more we explore the space of the parameters is much better to find suitable "contextual" fits to the output space. By exploring as many neighborhoods of local minimizers (while converging to local minimum) as possible, we could improve the training by starting our search from a local minimizer that worked for certain context. However, the optimizer would need to be smart enough to scape again from that local minimizer.

Okay, I found it (posted two days ago) using a link from the latest email received two hours ago (3/12 at 11:59AM).

Arben, Digest_Day5 is still not showing up in the Daily Q&A Digests. Did you by any chance put it in the Series 58 ChatGPT rather than the Series 62 ChatGPT? Thanks for the link. The three that I put in the other post are more about the geopolitical implications of AI and LLMs rather than their technology.

Ah, yes! The digest can be added asap, which at this hour probably means Monday morning :). As for an updated notebook—if you mean the transcript of the chats, I've added that already and it should be visible. Let me know if not! (If you mean the questions people asked in the Thursday survey, I'll try to review those more thoroughly and provide answers where reasonable.)

Arben, Can you please explain your decision in your communications to abrogate the standard rule of capitalizing the first letter in a sentence? Is that just a strategy to minimize the time to post a response to a question/comment? Is it a standard style followed by certain tech people? Would you do the same in more formal settings, such as published articles? Are you in the vanguard of a movement to change the English language? No disrespect or criticism intended - I am just curious …

Thanks Arbin, I did not see this aspect of the problem. Very interesting,