94
Textbook example for software safety harmed 6 people, LLMs killed 33 in two years.
(llmdeathcount.com)
this post was submitted on 27 May 2026
94 points (97.0% liked)
Fuck AI
7560 readers
745 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
To argue that ChatGPT et al. can't cause direct harm (i.e. irradiate people) misses the larger point I think. To me, that's slicing up the ethical responsibility of a system until no component bears ownership. Extending that logic to Therac-25 might be saying a function or variable can't irradiate people. Sure, not by itself. But it existed within a system that reliably produced harm.
From the Therac-25 wiki: "These accidents highlighted the dangers of software control of safety-critical systems."
I'd argue that mental health conversations are safety critical systems. Another safety critical system might be a basic search engine, if a user turns to it with real intent to do harm.
There's a reason we put safety warnings on products, regulate access to chemicals, etc, etc. I'm not arguing that we should do the same for LLMs, just showing that the Therac-25 example is rookie numbers compared to the reach and risks of modern software.
I feel like that largely ignores the idea of what an actual safety critical system is. It doesn't cover malicious usage in the browser case. Idk if I would put the amplifying effect it has on the mentally ill in that category either. It's just a wide net to be casting.
You would have to include stuff like social media, or just simply having conversations with people. Discord would fit in just as much as AI. Most of the internet would tbh.
It should probably be kept away from the mentally ill and children to an extent, but since it's completely virtual and available on the net, that would mean some intense regulation. No thanks, not interested in the internet being treated as a dangerous chemical. Regulations would help openai a lot more than regular folks in any case.