94
I Asked AI to Count My Carbs 27,000 Times. It Couldn’t Give Me the Same Answer Twice.
(www.diabettech.com)
this post was submitted on 01 May 2026
94 points (92.0% liked)
Technology
84257 readers
3895 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
They put lives at risk the same way every single product at your local home improvement store does. When you misuse a tool for a purpose it wasn't intended and isn't good at, you're going to get bad results.
This is an issue for the educational system, not the legal system.
Tools at home improvement stores were made to fulfill a specific purpose. GenAI still does not have a purpose it fulfills despite having hundreds of billions of dollars invested, not to mention all the other resources it's sucking up.
What if the packaging on every tool at home depot grossly misrepresented its capabilities and/or purpose?
This chainsaw cures cancer? Hot damn somebody call RFK!
Concrete mix goes great with pancakes, etc.
Does OpenAI claim ChatGPT is fit for those purposes? No.
The concrete itself will happily mix into your pancakes.
I think the whole point of this discussion is that the various peddlers of AI in fact do make wild claims about their capability.
My observation is that largely it's the downstream AI consumers who repackage it irresponsibly. That said, I don't hang on the words of Sam Altman and it's certain they are pushing the idea that AI is more capable than it is, but mostly what I see is them saying they built this thing and it does neat stuff and it can probably do neat stuff for you, use your imagination.
I believe a lot of the folks developing these tools would be horrified at the irresponsible ways vendors and end users are using it.
Sam Altman is the face of OpenAI. He is responsible for misrepresenting the product he sells. If you're going to sling blame around, then you had better observe the words of Sam Altman.
This sick man is taken seriously in mainstream media and politics, and it's no exaggeration to say he has blood on his hands.
As others have pointed out, this is also a problem with how they are advertising it.
If duct tape was advertised as something that you can use to hold your roof beams together, you'd have a issue with that.
And at the same time I wouldn't say "hey fuck that, duct tape is terrible! It doesn't hold beams together, I can't use it to tow a trailer, it's all just pretending to stick paper together because really every sliver of duct tape just sticks to the previous piece, etc etc" But that's the cool thing we do on Lemmy.
The ad is bad, duct tape ain't bad.
I have not seen OpenAI advertise ChatGPT as capable of medical diagnosis or therapy or anything like that. If you want therapy, and you can't afford better — because I think we can agree that AI is terrible at it, then there should be a therapy app with explicit safety controls.
The problem is someone created a screwdriver which is handy for lots of screwdriver shaped purposes and someone is trying to carve a ham.