Message Boards Message Boards

Ollama setup (post-it note instructions)

Posted 2 days ago

Set a Windows User Environment Variable OLLAMA_ORIGINS with a value of * (Start -> Settings -> System -> About -> Advanced System Settings (on the right) -> Advanced (tab) -> Environment Variables -> Add a new one on top)

Here's your HTTP request. The "ExportString" is the magic:

ollamaHttpRequest[model_, contextSize_, context_, seed_, streaming_, 
       prompt_] := HTTPRequest["http://127.0.0.1:11434/api/generate",
       <|"Method" -> "POST",
        "Headers" ->
         <|"Content-Type" -> "application/json"|>,
        "Body" -> ExportString[
          <|"model" -> model,
           "prompt" -> prompt,
           "context" -> context,
           "stream" -> streaming,
           "options" ->
            <|"seed" -> seed,
             "num_ctx" -> contextSize|>|>,
          "JSON"]|>];

Do a URLRead on that to send it to Ollama and you get an object from Ollama you can parse (should give you a '200' status server code).

Here's how you parse a non-streaming response:

ollamaReadSingle[ollamaHttpResponse_] := 
  Association[ImportString[ollamaHttpResponse["Body"], "JSON"]];

And here's how you parse a streaming one:

ollamaReadStreaming[ollamaHttpResponse_] := 
  Association[#] &@ImportString[#, "JSON"] & /@ 
   StringSplit[ollamaHttpResponse["Body"], "\n"];

Confirming this doesn't to async responses yet (so you will wait even if you put in "streaming" unless you figure that part out). Didn't see this anywhere else. As much fun as it is being the only person in the world with a barely-working prototype...

POSTED BY: Cameron Kosina
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract