this post was submitted on 22 Sep 2026
1244 points (98.5% liked)
Funny
16292 readers
834 users here now
General rules:
- Be kind.
- All posts must make an attempt to be funny.
- Obey the general sh.itjust.works instance rules.
- No politics or political figures. There are plenty of other politics communities to choose from.
- Don't post anything grotesque or potentially illegal. Examples include pornography, gore, animal cruelty, inappropriate jokes involving kids, etc.
Exceptions may be made at the discretion of the mods.
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The LLM itself is deterministic. It outputs a vector that we interpret as a probability distribution over the set of tokens. It's the program using the output of the LLM, such as a chatbot program, that selects an individual token using (or not using) these vector elements as weights.
The weights do not change once the model is trained. This is why I am suggesting they could be incorporated directly into the structure of an ASIC for a specific model, rather than storing them in memory.
Edit: Of course another major factor could be that the models are just to big to be wholly implemented in a single IC by any currently existing manufacturer.