Message Boards Message Boards

2
|
1943 Views
|
1 Reply
|
4 Total Likes
View groups...
Share
Share this post:

Using connection accept Server-events (SSE) with OpenAILink`

Has anyone implemented a connection receiving server-events?

As described in https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format

That's what is being used by chatGPT to update the response. But is it used by many servers to update browser during in long compute.

It would be nice to be able to use with OpenAILink` ...

POSTED BY: Luc Barthelet

I was able to make this work using URLSubmit

body = <|
                           "model" -> "gpt-3.5-turbo",

   "messages" -> {<|"role" -> "user", 
      "content" -> "Can you give me a short description of Etiopia?"|>},
                           "n" -> 1,
                           "temperature" -> 0.2,
                           "stream" -> True
                       |>;

bodyRule =
              If[body === None,
                   Nothing,
                   "Body" -> ExportByteArray[body, "JSON"]
               ];

request = HTTPRequest[
               <|
                    "Scheme" -> "https",
                    "Domain" -> "api.openai.com",
                    "Method" -> If[body === None, "GET", "POST"],
                    "ContentType" -> "application/json",
                    "Path" -> {"v1", "chat", "completions"},
                    bodyRule,
                    "Headers" -> {
                          "Authorization" -> "Bearer " <> apiKey
                      }
                |>
           ];
receiveBodyChunk[partialResponse_] := 
  Module[{chunks, chunkText, validChunks},
   chunkText = ImportString[partialResponse["BodyChunk"], "Text"];
   chunks = StringSplit[chunkText, "\n"];
   validChunks = 
    ImportString[StringDrop[#, 5], "JSON"] & /@ 
     Select[ chunks, StringLength[#] > 5 && # != "data: [DONE]" &];
   chunkText = Cases[validChunks, ("content" -> l_String) :> l, -1];
   BodyString = BodyString <> (StringJoin @@ chunkText)
   ];

BodyString = "";
Print[Dynamic[BodyString]];
response2 = 
 URLSubmit[request, 
  HandlerFunctions -> <|"BodyChunkReceived" -> receiveBodyChunk|>, 
  HandlerFunctionsKeys -> {"BodyChunk"}]
POSTED BY: Luc Barthelet
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract