Message Boards Message Boards

GROUPS:
Staff Picks Data Science Image Processing Graphics and Visualization Wolfram Language Statistics and Probability Machine Learning Natural Language Processing Know-How Computational Humanities Neural Networks
Jofre Espigule-Pons

Shakespearean GPT from scratch: create a generative pre-trained transformer

Jofre Espigule-Pons, Wolfram Research
Posted 19 days ago
1178 Views
|
8 Replies
|
24 Total Likes
|

enter image description here
POSTED BY: Jofre Espigule-Pons
8 Replies
Sort By:

I find using the OpenAILink within the wolfram notebook kinda fun. And the ability to save them in an orderly fashion super useful. I can almost write sort of a graphic novella. Here's my first try with the defaults enter image description here

enter image description here However, the OpenAICompletion did not follow the 7 word and 5 line rule that I gave. it worked fine on the openAI website but not with the packet command. Here's the output from chatGPT

enter image description here

I tried to change the OpenAIMethod with no luck

enter image description here
POSTED BY: Jack I Houng

After reading "What Is ChatGPT Doing … and Why Does It Work?" I really wanted to explore the networks, better understand what they do and play around with them in Mathematica. This post gave me everything in one go... which is awesome!

POSTED BY: Martijn Froeling

Happy to hear that. :)

I'm preparing another post on the original vanilla encoder-decoder transformer architecture (Attention is all you need paper from 2017).

It also might be of your interest this other recent post on Few Shot Learning using GPT-3 and WL: https://community.wolfram.com/groups/-/m/t/2848741
POSTED BY: Jofre Espigule-Pons

This is an excellent hands-on presentation. Thank you for this. I do have a question, though. In the example with:

trainedNet=NetExtract[results["TrainedNet"],"decode"];

Instead of using CloudPut and CloudGet, what would be the correct expression to save to a local file and retrieve it from the local file?
POSTED BY: Loren Abdulezer

Happy to hear that you enjoy it. :)

You can use Export, for example:

Export["trainedModel.wlnet", trainedNet]

And then you can get the model with Import. Note that you might want to specify a particular folder directory to store the model. For example with SetDirectory.

Hope this helps.
POSTED BY: Jofre Espigule-Pons

Works perfectly. Thanks!
POSTED BY: Loren Abdulezer

Excellent introduction ... thanks!
POSTED BY: Stuart Nettleton

enter image description here -- you have earned Featured Contributor Badge enter image description here Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!
POSTED BY: Moderation Team
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract