When editing the selected model you can either: edit at the cell scope (what you're OP is mentioning) or edit at the notebook scope

To edit it for the selected notebook, you want to edit the model selection through the toolbar.

There's a pull request to make this selection a bit more intuitive globally (AKA a default state for any session) here: https://github.com/WolframResearch/Chatbook/pull/304 Before that gets merged and pushed out to users, there's a programmatic way you could set this so that any new chat notebook uses gpt-4 CurrentValue[ $FrontEnd, { TaggingRules, "ChatNotebookSettings", "Model" } ] = "gpt-4”

Try this, for now:

CurrentValue[$FrontEnd, TaggingRules] = GeneralUtilities`ToAssociations@CurrentValue[$FrontEnd, TaggingRules];
CurrentValue[$FrontEnd, {TaggingRules, "ChatNotebookSettings", "Model"}] = "gpt-4"

See Avery's comment for a more comprehensive answer. My code snippet will set "gpt-4" once and for all (at the $FrontEnd level, which is persistent across frontend notebook sessions)