Group Abstract Group Abstract

Message Boards Message Boards

4
|
3.7K Views
|
18 Replies
|
15 Total Likes
View groups...
Share
Share this post:
GROUPS:

[WSG25] Daily Study Group: What is ChatGPT Doing... and Why Does It Work?

Posted 5 months ago
POSTED BY: Arben Kalziqi
18 Replies
Posted 4 months ago
POSTED BY: Updating Name
Posted 5 months ago
POSTED BY: Phil Earnhardt
Posted 5 months ago

Arben, On Wednesday I had asked, "Is DeepSeek doing something fundamentally different, or did they find a way to do it more efficiently?" You responded, "I have to say I'm not sure on that front—I'll try to look into this tomorrow and post about it in the Community thread." Have you had a chance to look into this yet?

Also, I am not seeing a Digest_Day5 in the Daily Q&A Digests.

POSTED BY: Gerald Oberg

Hmm... annoying!! I absolutely dropped it in there yesterday. I'll make sure it works this time.

As far as DeepSeek goes, as best I can tell there are indeed some architectural improvements, but the overall idea and structure remain the same and most of the improvements are in the training process. This article from MIT Technology Review is insightful! https://www.technologyreview.com/2025/01/31/1110740/how-deepseek-ripped-up-the-ai-playbook-and-why-everyones-going-to-follow-it/

POSTED BY: Arben Kalziqi
Posted 5 months ago

Arben, Digest_Day5 is still not showing up in the Daily Q&A Digests. Did you by any chance put it in the Series 58 ChatGPT rather than the Series 62 ChatGPT? Thanks for the link. The three that I put in the other post are more about the geopolitical implications of AI and LLMs rather than their technology.

POSTED BY: Gerald Oberg

Added!! Sorry about that!

POSTED BY: Arben Kalziqi
Posted 5 months ago

Okay, I found it (posted two days ago) using a link from the latest email received two hours ago (3/12 at 11:59AM).

POSTED BY: Gerald Oberg

Arben, Thanks 10^6 for a wonderful course! Question: Will you be posting an updated Q&A Digest along with an updated notebook for Day 5? We would appreciate it very much.

POSTED BY: Zbigniew Kabala

Ah, yes! The digest can be added asap, which at this hour probably means Monday morning :). As for an updated notebook—if you mean the transcript of the chats, I've added that already and it should be visible. Let me know if not! (If you mean the questions people asked in the Thursday survey, I'll try to review those more thoroughly and provide answers where reasonable.)

POSTED BY: Arben Kalziqi
Posted 5 months ago

I was wondering in the last class two topics related to line of research in Wolfram:

1) Is Wolfram Research developing their own Algorithmic Differentiation framework for programs, a.k.a (parametrized)functions, that depends on loops, conditionals, recursion, and (non)smooth elemental functions? I commented in class this is between numerical and symbolic differentiation. By the way, this is the standard in Scientific Machine Learning and the backbone to make computationally efficient optimizers like ADAM (and their subsequent versions).

2) In case that (1) is negative, are you more focused on developing the discrete techniques to emulate neural networks capacity via Rule Arrays applied to cellular automata?

POSTED BY: Angel Rojas
Posted 5 months ago

Arben, Can you please explain your decision in your communications to abrogate the standard rule of capitalizing the first letter in a sentence? Is that just a strategy to minimize the time to post a response to a question/comment? Is it a standard style followed by certain tech people? Would you do the same in more formal settings, such as published articles? Are you in the vanguard of a movement to change the English language? No disrespect or criticism intended - I am just curious …

POSTED BY: Gerald Oberg

I don't! If you're referring to the Q&A digests, that's because those are just logs of the live chats from the sessions rather than email or otherwise "official"/formal communications. I think you'll find that the number of people who capitalized the first word of their messages back on AIM in 1997 was also quite small—though to your point, I do imagine that that number is shrinking over time as people have more access to instant back-and-forth communication. Language does change over time, and while I am largely a stickler for rules in a visceral sense I'm certainly not a prescriptivist. (If I were, I might point out that you use a hyphen rather than an em-dash in your last sentence, and add a novel space before the ellipses :). Language—written and spoken—always changes, particularly when exacerbated by the movement to new mediums and entry tools like keyboards where it's easier to type a hyphen than an en- or em-dash.)

POSTED BY: Arben Kalziqi
Posted 5 months ago

In response to the survey question, "How will you use what you learned at this Daily Study Group?" I wrote: I will have a better appreciation and understanding of what is being discussed in the numerous newscasts, podcasts, articles, interviews that one encounters about ChatGPT or other LLMs. Something the course did not address: How could the technology overviewed lead to the catastrophic results people are predicting about AGI? One would not think that even monstrously large matrices could produce malicious consciousness. (I am sometimes reminded of the book I read as a teenager, "Colossus: The Forbin Project", a 1966 science fiction novel by D. F. Jones, about super-computers taking control of mankind.) The things Arben demoed are really impressive, but there is no "mind" (with potential intensions) producing them. I would like to hear Arben's thoughts about these issues. Even more, can you point us somewhere that Stephen Wolfram has discussed these issues (potential threats of AI or AGI)?

Here is a good expert discussion: https://drive.google.com/file/d/1JVPc3ObMP1L2a53T5LA1xxKXM6DAwEiC/view

Also: https://www.google.com/books/edition/The_Age_of_AI/Y2QwEAAAQBAJ?hl=en&gbpv=1

This could be considered a post-DeepSeek update to the reference above: https://www.csis.org/analysis/deepseek-huawei-export-controls-and-future-us-china-ai-race

POSTED BY: Gerald Oberg
POSTED BY: Arben Kalziqi

Thanks Arbin, I did not see this aspect of the problem. Very interesting,

POSTED BY: Laurence Bloxham
POSTED BY: Laurence Bloxham
POSTED BY: Arben Kalziqi
Posted 5 months ago
POSTED BY: Angel Rojas
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard