this post was submitted on 19 Aug 2025
346 points (96.3% liked)
Technology
74233 readers
4349 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
It annoys me that Chat GPT flat out lies to you when it doesn’t know the answer, and doesn’t have any system in place to admit it isn’t sure about something. It just makes it up and tells you like it’s fact.
LLMs don't have any awareness of their internal state, so there's no way for them to see something as a gap of knowledge.
Wouldn't it make sense for an ai to provide a confidence level though?
I've got 3 million bits of info on this topic but only 4 of them lead to this solution. Confidence level =1.5%
It's always funny to me when people do add 'confidence scores' to LLMs, because it always amounts to just adding 'say how confident you are with low, medium or high in your response' to th prompt, and then you have made up confidences for made up replies. And you can tell clients that it's just made up and not actual confidence, but they will insist that they need it anyways…
It doesn't have "3 million bits of info" on a specific topic, or even if it did, it wouldn't be able to directly measure it. It's worth reading a bit about how LLMs work behind the hood, because although somewhat dense if you're new to the concepts, you come out knowing a lot more about what to expect when using them, what the limitations actually are and how to use them better if you decide to go that route.
You could do this with logprobs. The language model itself has basically no real insight into its confidence but there's more that you can get out of the model besides just the text.
The problem is that those probabilities are really "how confident are you that this text should come next in this conversation" not "how confident are you that this text is true/accurate." It's a fundamental limitation at the moment I think.