Another task I've had success with is "make this code faster". It's sometimes able to rewrite the inefficient For loops in terms of vectorized matrix expressions

I asked GPT-4 to "write a short tutorial with examples of CombinedEntityClass in the Wolfram Language. The result looked believable, but it was mostly made-up: EntityClass expression were incomplete, some Entity types non-existent etc. A few code examples are shown below.

usStates = EntityClass["AdministrativeDivision", "UnitedStatesStates"];
canadaProvinces = 
  EntityClass["AdministrativeDivision", "CanadaProvinces"];
combinedClass = CombinedEntityClass[{usStates, canadaProvinces}];

instruments = EntityClass["MusicalInstrument"];
manufacturers = 
  EntityClass["Organization", "MusicalInstrumentManufacturers"];
combinedClass = CombinedEntityClass[{instruments, manufacturers}];

    physicsLaureates = EntityClass["Person", "NobelPrizePhysicsLaureates"];
chemistryLaureates = 
  EntityClass["Person", "NobelPrizeChemistryLaureates"];
combinedClass = 
  CombinedEntityClass[{physicsLaureates, chemistryLaureates}];

I think GPT-4 needs a Wolfram-U online course first :)

The main challenge I have found is getting ChatGPT 4 to produce correct vectorized code. Even given quite short procedural code snippets, it fails in repeated attempts to vectorize them correctly.

This isn't a failing specific to the Wolfram language, however: the same thing occurs with Python and Matlab.

What I am hoping is that Wolfram will eventually provide api connectivity between ChatGPT 4 and the Wolfram documentation library, which is organized very systematically and is replete with examples of function usage. Given this, and an ability to test its own code, ChatGPT should be able to improve its performance with WL programming, over time.

I've had about 50% rate of success at correct vectorization. One of the issues is that transformer models use the same amount of compute per prompt. Hence, harder questions need several prompts to get right.

Yes.

1. Install the paclets "OpenAILink" and "OpenAIMode".
2. Follow the setup instructions in "OpanAILink".
3. Run Needs["AntonAntonovOpenAIMode"].
4. Run OpenAIMode[] in a desired notebook

More details are given in this discussion.