Group Abstract Group Abstract

Message Boards Message Boards

Running a local LLM using llamafile and Wolfram Language

Jon McLoone, Wolfram Research
Posted 2 years ago

Attachments:
DOWLOAD-DESKTOP-...nb
POSTED BY: Jon McLoone
9 Replies
Sort By:

Local LLM may be supported in v14.3. enter image description here
POSTED BY: Kotaro Okazaki
POSTED BY: Jon McLoone
Dave Middleton
Dave Middleton
Posted 2 years ago
POSTED BY: Dave Middleton

Indeed, it was suggested at the last Tech Conference that 14.2 would have local LLM support, but this doesn't seem to be reflected in the documentation. (Haven't had the chance to upgrade yet)...
POSTED BY: Joshua Schrier
Francesco S
Francesco S
Posted 11 months ago

Anything useful along those lines available in 14.2 ? Thank you.
POSTED BY: Francesco S
Jacob Evans
Jacob Evans
Posted 1 year ago

This is great news. I would love to "play" with Liama 3 locally using the built-in LLM functions I've already used for performing some of my use cases.
POSTED BY: Jacob Evans

Interesting that's good to know! To be honest calling the server was interesting as there are many systems that now build API by servers (like ollama for Mac).

I guess I'll have to give my bucks to OpenAI for a while then!!

POSTED BY: Ettore Mariotti

Hi Jon! This is very interesting.

I was wondering if it's possible to configure a local language model as the default large language model (LLM) that the system uses when dealing with features like LLMFunction, etc.

It would be great if it were possible to leverage all the Wolfram Language technology already developed for LLMs, such as the prompt repository, and so on.

Does anyone have any idea about the feasibility of this?
POSTED BY: Ettore Mariotti

enter image description here -- you have earned Featured Contributor Badge enter image description here Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!
POSTED BY: EDITORIAL BOARD
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard