Thanks for this. Been hacking away in Mathematica for all these reasons... it's promising for a lot of reasons and then it's also adding to the computational irreducibility.

I think its biggest value is going to be as "glue" between formal systems. At the end of the day... the entity system for Wolfram Data and the SPARQL stuff with wikidata kinda shows how much we require hybrid approaches.

Beyond that, there are some things that just going to be inefficient to express even in few shot gpt3 formulations.

But, just like Wolfram|Alpha has dramatically improved productivity this should help on some things.

Hi Sam, you ask a good question.

That's the surprising part -- the only WL "training data" given to the model is the gray text shown in each example above. This is sometimes referred to as "few shot learning" -- when a model is able to adapt to a new task by only seeing a few examples.

That said, OpenAI trained GPT-3 on a huge corpus of text taken from the web, which undoubtedly would have contained examples of WL code, so it's possible that's it's drawing on some of that prior knowledge.

In each case, the final output (which is in bold) is prepended with a number of examples. GPT generalizes from those examples, which indicates which language it is working in. That "prompt," plus its training data, is enough to let it figure out the answers.

Wolfram Alpha could definitely integrate GPT-3. AI Dungeon is a commercial product (game) that has done so.

https://twitter.com/nickwalton00/status/1283984454952693760?s=19 :

Yes there API is available for commercial use (their goal is for it to be a commercial product) and there aren't explicit rate limits per say, but you'd need to have a conversation before hand if you wanted to push through large amounts of traffic.

I suppose this is how the free-form input feature will function eventually. Unless it already uses machine learning ?