94
I Asked AI to Count My Carbs 27,000 Times. It Couldn’t Give Me the Same Answer Twice.
(www.diabettech.com)
this post was submitted on 01 May 2026
94 points (92.0% liked)
Technology
84257 readers
3895 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Bruh a couple of months ago I asked it (Gemini) to check the number of characters, including spaces, in a potential game character name because I was working at the time and couldn't stop to check my in-head count. It told me 21--I had counted 20. I thought I must have gotten distracted and miscounted. Later when I had time to actually focus on the issue it turned out AI had miscounted a 20 character string (maybe counting the null terminating character?).
AI doesn't see individual characters, it sees tokens, with most tokens being a word or part of a word. That's why per-character questions have such a high failure rate.
It's it doesn't understand the simple concept of the number of letters and spaces, it needs to be reprogrammed.
How many letters are there in 令牌? It's a simple question right, you wouldnt need to search for it to find out would you?
It doesn't understand anything though? It never will. It's a probability machine. If you choose to believe its output, that's on you. I use it as a coding assistant to get boring things done faster. Fire a prompt at claude code, grab a coffee, check out the diff. But that last step is crucial. Can't trust AI output blindly.
The embedding layer post tokenization is not just a probability machine the way you're suggesting it. You can argue that it is probabilistic with inferred sentiment, but too many people think it works like how text prediction on your phone does and that is just factually inaccurate.
Verify output of course, but saying "it doesn't understand anything" and "probability machine" is a borderline erroneous short sell. At the level of tokens it "understands" relationships, and those relationships are not probabilistic, though they are fundamentally approximated based on a training corpus.
Can you explain how it's more than probability? It's using a neural network to guess the most likely next token, isn't it?
You could also say that it chooses what will be the next word it will say to you. It has a few words to choose from, which it has selected in relation to the previously spoken words, your question and previous interactions (the context). The probability you're talking about (a number) could also be seen as it's preference among those words. I'm not sure the probability vocabulary/analogy is necessarily the best one. The best might be to not employ any analogy at all, but then you have to dig deeper into the subject to form yourself an informed opinion. This series of videos explains it better than I do : https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi