Group Abstract Group Abstract

Message Boards Message Boards

8
|
7.5K Views
|
30 Replies
|
32 Total Likes
View groups...
Share
Share this post:
GROUPS:

[WSG24] Daily Study Group: What is ChatGPT Doing... and Why Does It Work?

A one-week Wolfram U Daily Study Group covering Stephen Wolfram's best-selling book What is ChatGPT Doing... and Why Does It Work? begins on Monday, September 9, 2024.

enter image description here

Join a cohort of fellow learners to discover the principles that underly ChatGPT and other LLMs. I have adapted the material from the aforementioned book into a series of four notebooks covering topics ranging from probabilistic text generation to neural nets, machine learning, embeddings and even transformer models.

Giulio Alessandrini, our manager of machine learning at Wolfram, will join to answer your questions on the day we cover neural networks. On the final day of the study group, Alan Joyce—who directs content development at Wolfram|Alpha—will join us to explain how the computational powers of Wolfram Language can be integrated with ChatGPT, and why this integration is much more than the sum of its parts.

Stephen Wolfram's book is aimed at anybody who is curious about these ideas, and this study group follows the book's lead. Therefore, no prior Wolfram Language, machine learning, or even coding experience is necessary to attend this study group.

Please feel free to post any questions, ideas and/or useful links in this thread between sessions—we always love to continue the discussion here on Community!

This is a one-week study group that will run from September 9 through September 13, 2024 at 11:00am Central US Time each day.

REGISTER HERE

enter image description here

POSTED BY: Arben Kalziqi
30 Replies

What other software language does the similar to Wolfram language to synthesizing with ChatGPT?

POSTED BY: Taiboo Song

Honestly, I'm not aware of any. That doesn't mean that they don't exist, but if they do, I don't think I've heard of them. Now, ChatGPT can write code in a bunch of different languages, and you can train models to learn different programming languages, but as far as "providing information about the world from strictly factual sources and enabling more general computation", I don't think there's any other one.

POSTED BY: Arben Kalziqi

Awesome class and now understand the ChatGPT under the hood. It is really amazing that Steven Wolfram already knows the future and started using neural networks early. How did Steven get into using neural networks? Any good course that you recommend to learn about neural network?

POSTED BY: Taiboo Song
POSTED BY: Arben Kalziqi

Thank you Arben. In this scenario do you think the neural net could start to memorize the traffic information? If that happen I starting to see the problems that would trigger.

I think it could, but generally the problem of memorization is not too serious a one in modern neural net development, is my understanding. The ideal would be that the net builds a good model of what traffic flow looks like, and as long as there's a good amount of data it should be able to account for the fact that traffic has periods of extreme irregularity (e.g. crashes or construction). If the designers realize that it's doing less modeling and more memorizing, they can change the training to account for that (and typically, it's pretty easy to realize during the training process whether a net is "overfitting" or memorizing).

POSTED BY: Arben Kalziqi

Of course! My background is in physics, so while this stuff is maybe a little closer to home, I was really impressed when I learned a bit about how it works and how "simple" everything is on the inside, mechanically speaking.

POSTED BY: Arben Kalziqi
POSTED BY: Arben Kalziqi

Thanks for the clarifications! And congrats, your presentations in the study group were excellent.

Thank you very much! I'm glad I was able to properly convey the ideas outlined in Stephen's book.

POSTED BY: Arben Kalziqi
Posted 1 year ago

Reinforcement learning, maybe?

POSTED BY: Héctor Galaz
POSTED BY: Carl Hahn

Hey Carl—there shouldn't be any issues with either way of doing things, and they should indeed be equivalent. (Now, Day 3's notebook is so big that it might cause an issue here or there, but even that one ought to be fine.)

POSTED BY: Arben Kalziqi

AI has so many ways to describe it: AI, Machine Learning, Deep Learning, and Generative AI. Which term is the best way to describe Wolfram's approach? Why is Wolfram's purpose to get into AI world?

POSTED BY: Taiboo Song
POSTED BY: Arben Kalziqi
POSTED BY: Arben Kalziqi

I heard AI for more than 40 years and it seems like a smart design? Are there any major differences?

POSTED BY: Taiboo Song
POSTED BY: Arben Kalziqi

Hey Arben, I have already learned more than I expected to from Wolfram's perspective on AI, Machine Learning and ChatGPT. Looking forward to your approach to teaching.

But could I really use Wolfram Language alone, or WL combined with (?), to do industrial grade things like spot defects in images of circuit boards or components soldered down on circuit boards? Like cracks, malformed parts, missing traces, debris, etc? Or would that still require developing a specialized tool? Or look at images of objects from different angles and determine if they are images of the same thing? Like an aerial photo of a city or a farm?

Or better yet, after recognizing whether they are images of the same thing, highlight what has changed in the thing after correcting for the different look angles and lighting? Or would that be too specialized an application?

(I am certain that if I wanted to do that I would at least look at WL as my experimental laboratory for learning how to do it)

POSTED BY: Carl Hahn
POSTED BY: Arben Kalziqi
Posted 1 year ago

I was curious to see that the sample from the 4-gram model in today's session/notebook was the smallest to generate actual English words. Is that likely related to the fact that 4.7 letters is the average word length in English? (Source)

POSTED BY: Henry Ward
Posted 1 year ago

My mistake, the 2-gram generated 'wore' and 'hi' and the 3-gram generated 'was'. Though I imagine the increase in the proportion of valid words in the sample is related to the distribution of word lengths in English?

POSTED BY: Henry Ward

That may well be true. For the words -> sentences analogy, sentences can have extremely variable lengths, so it might not just be the mean word length but the tightness of that distribution!

POSTED BY: Arben Kalziqi
Posted 1 year ago

Arden! I'm the first to plant my flag! Yay!!!!

I got the dark them working! Look, Mitch didn't register, lol :D

Attachment

Attachments:
POSTED BY: Tingting Zhao

Try ImageIdentify[pic,All,3], perhaps :). (Also, it's "Arben" with a "b"!)

POSTED BY: Arben Kalziqi
Posted 1 year ago
Attachment

Attachments:
POSTED BY: Tingting Zhao
Posted 1 year ago

I tried 10

Attachment

Attachments:
POSTED BY: Tingting Zhao
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard