Group Abstract Group Abstract

Message Boards Message Boards

[WSS23] Study of the possibility of phase transitions in LLMs

2 Replies

This is a great project. I'm curious to see how these phase transitions differ between quantization and model type.

I'd imagine that different models would have different 'temperatures' for their phase states. I imagine this is compounded by the noise introduced via quantization and even more so when discussing different transformer types.

POSTED BY: Nicholas Scott
POSTED BY: EDITORIAL BOARD
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard