Something like this.
callOpenAIStreaming[ messages_List, model_String : $OpenAIModel] :=
Module[{requestBody, response, retryCount = 0, responseData},
With[{
endpoint = "https://api.openai.com/v1/chat/completions"
},
$OpenAIChatCompletionResponse = "";
requestBody =
ExportString[{"model" -> model, "messages" -> messages,
"temperature" -> 0.3, "stop" -> Null, "stream" -> True}, "JSON"];
TaskWait[
URLSubmit[
HTTPRequest[endpoint,
Association["Method" -> "POST",
"Headers" -> {"Content-Type" -> "application/json",
"Authorization" -> "Bearer " <> $OpenAISecretAPIKey},
"Body" -> requestBody]],
HandlerFunctions -> <|
"BodyChunkReceived" -> processChatCompetionChunk|>,
HandlerFunctionsKeys -> {"BodyChunk"}
]];
Return[$OpenAIChatCompletionResponse]
]
]
Set up a global variable. Not nice but we're already doing globals for the model and the API secret key. You an use this to display the response as it is streamed:
Dynamic[StringRiffle[TextSentences[$OpenAIChatCompletionResponse],
"\n"]]
The trick is to use URLSubmit[] and write a handler for "BodyChunkReceived".
You really don't want async execution here. You can use TaskWait[] and wait for the entire response come back from ChatGPT and then return the global variable.
I parsed the JSON and it works, but I think my code is ugly... interested in how others do it.
