this post was submitted on 03 Jun 2026
964 points (99.7% liked)
People Twitter
10414 readers
402 users here now
People tweeting stuff. We allow tweets from anyone.
RULES:
- Mark NSFW content.
- No doxxing people.
- Must be a pic of the tweet or similar. No direct links to the tweet.
- No international politcs
- Be excellent to each other.
- Provide an archived link to the tweet (or similar) being shown if it's a major celebrity, figure or any politician. See stickied post https://web.archive.org/save is an option. Archive.is is no longer recommended because they remove/alter content.
- If a tweet (or similar) makes a statement of fact, either mark it as "satire" in the body or provide the original source link backing up the statement (see rule 6)
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
No, it is cheap and efficient. It is relative, and the comparison is to model training. But yeah, its not free
Sure it's much much cheaper than training, but importantly those companies are not recouping anything with inference because it is still more expensive than what they are selling it for.
They are double bankrupting themselves.
At work we run inference for a research project with an open weights model in the public cloud another part of my company provides and we pay around 25$ a day for a VM with a single L40s. It's both slow - despite not even serving concurrent users - and kind of bad in its outputs.
Edit: Interference -> Inference, arguing on the internet after waking up first thing in the morning might not have been the best idea