wow, chat notebooks work nicely. So far I couldn't add my own persona (prompt resource) to the list.

If I define an LLM Tool... how can I output something and make it appear underneath the chat output cell? I've tried CellPrint but its output goes to the Messages view instead of the current notebook. What's the trick? It would be very useful to define tools that could use the full wolfram functionality for output so in addition to the text generated by the LLM we would have interactive maps, graphs, diagrams etc.

The speech bubble is the same mechanism you see in a chat-driven or chat-enabled notebook after hitting ' to open a new ChatInput cell. If you're in a notebook that is not a Chat Notebook, from that toolbar speech bubble you can hit the Enable AI Chat Features checkbox to turn it into one. If your paclet is out of date, it will prompt you to install the latest version like the image below. You may have to restart the front end / application after an update.

You shouldn't need to create any initialization cells for a prompt or LLMTool using that definition notebook

Yes it's still being worked on. Here's an example that uses the LLMTool documentation example of population finding. You can add it as a persona to chat notebooks by deploying to the cloud and adding it from the add/manage persona menu by selecting to use a URL.

Edit: I should add that with chat notebooks tool calling only works with gpt-4 right now, gpt-3.5-turbo makes mistakes. (We have looked into the function calling model but have come across hallucinations it makes thinking it has access to python functions it can call that don't exist.) You can use them programmatically with LLMSynthesize, etc. by setting your LLMEvaluator to have access to tools.

To do the same with the nb I provided it'd be

LLMSynthesize["What is the population of Champaign,IL?", "FullText", 
 LLMEvaluator -> <|
   "Tools" -> 
    LLMTool["cityPopulationFinder", 
     "city" -> "City", #city["Population"] &]|>]

Example in a chat notebook