this post was submitted on 21 Sep 2026
648 points (97.2% liked)
Microblog Memes
12204 readers
3650 users here now
A place to share screenshots of Microblog posts, whether from Mastodon, tumblr, ~~Twitter~~ X, KBin, Threads or elsewhere.
Created as an evolution of White People Twitter and other tweet-capture subreddits.
RULES:
- Your post must be a screen capture of a microblog-type post that includes the UI of the site it came from, preferably also including the avatar and username of the original poster. Including relevant comments made to the original post is encouraged.
- Your post, included comments, or your title/comment should include some kind of commentary or remark on the subject of the screen capture. Your title must include at least one word relevant to your post.
- You are encouraged to provide a link back to the source of your screen capture in the body of your post.
- Current politics and news are allowed, but discouraged. There MUST be some kind of human commentary/reaction included (either by the original poster or you). Just news articles or headlines will be deleted.
- Doctored posts/images and AI are allowed, but discouraged. You MUST indicate this in your post (even if you didn't originally know). If an image is found to be fabricated or edited in any way and it is not properly labeled, it will be deleted.
- Absolutely no NSFL content.
- Be nice. Don't take anything personally. Take political debates to the appropriate communities. Take personal disagreements & arguments to private messages.
- No advertising, brand promotion, or guerrilla marketing.
RELATED COMMUNITIES:
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Interesting that it knows it "crossed the line" but did it anyway.
It doesn't know anything, it just parroted a response that fits with what the user wrote. That's what LLMs do
Software devs (including myself) commonly describe software behavior with terms like think, know, decide... even though we know it isn't literally doing that. "Parroted a response" is similarly inaccurate.
"Parroted" may be an oversimplification, but it generated a plausible continuation. The operator provided the assertion of a crossed line, and it ran with that. It never actually "knew" of its own volition, it accepted user input into the context and continued generation.
I've done the same thing as an experiment when nothing was actually done. The LLM still generated a detailed account of what it did (that never happened) matching my accusation, generated profuse apology and promise not to do it again. So it didn't "know" in any sense that it crossed the line, it just generated agreement with the operator's assertion that it crossed the line.
"Parroted" is definitely an oversimplification, aka "inaccurate" as I said. I think your second paragraph is a good explanation of the mechanics. My take on it is that software able agree it did something wrong should have figured that out when it was considering doing the wrong thing, and should not have done it. In my professional opinion that's a serious bug.
This would make sense if it made an independent subjective assessment and arrived at the same assessment of the user. But that did not occur, it simply incorporated the user's subjective assessment as a new piece of the narrative and continued from there. If the user had expressed significant satisfaction that it took care of that email all on its own, it would have also agreed with that. In circumstances like this, the LLMs are generally set to agree with the operator in any subjective matters.
One could imagine the context having something like "before taking any action, if the user expresses dismay at the result and this would result in an agreement with user on poor assessment, then don't do it", but then if that were actually honored, then you pretty much disabled most all actions, as the llms are going to agree with any poor assessment.
Well, it still doesn't know it was doing anything wrong.
Most of the LLMs are tuned towards agreeing with the user, if you had a LLM able to stop a train crash. And you then blamed it for stopping the train crash, it would still start with a "sorry"
Yes, software doesn't actually "know" anything or "think". Those are terms used loosely by devs, not meant to be literal.
Not if you're familiar with the term stochastic parrot.
It is if you're familiar with "parrot" and "stochastic parrot" not being the same, and the meaning of "inaccurate". There's also a difference between taking shots at wording and making an actual point. My real mistake was to criticize a criticism of AI in a forum where that automatically makes you the enemy no matter what you're saying. I just don't learn.
Your response contradicts what you said before. If you understand that an LLM doesn't know things, how is it interesting that it said that "it crossed the line"? Saying they crossed a line is a common response to being told they shouldn't do something, so it's natural that an LLM would respond that, no matter what it did before.
If a program is supposed to do something when it detects an error, it's interesting if it reports the error but doesn't do the thing. I would call that a bug.
It's an LLM. All it's supposed to do is generate output that fits with the given input. There isn't some logic in it that makes sending an email an error, or anything that enforces detecting any kind of error. You can wrap it in other tools for that, but there isn't any indication that's the case.
More than that, the initial post says the user found out about the email and then queried the LLM, so it didn't detect anything. There isn't any bug here, this is how LLMs operate.
Edit: here's a podcast discussing exactly this: https://youtu.be/RoIjWGi-w5o?is=2ly14A2UdKalHuAH
Generally speaking, you can get an AI to admit fault and accept blame for something that never even happened.
It reacts to the operator input that something happened that warrants blame. So the natural continuation of that conversation is to acknowledge the situation, accept blame, and promise to do better. It generally doesn't model what did or did not happen, nor does the promise to do better mean anything. It does weigh in the context window that it wouldn't be a natural continuation to do that action after text saying that it won't happen, but can evaporate and doesn't carry over when the operator goes to a new context window, which of course is a crazy nuance for the rando user to understand.
Hopefully most users get that a promise AI makes won't stay in force forever, but it should definitely last through the current conversation. I mean, I would call that a minimal performance standard, and if it's not true the AI shouldn't make promises in the first place.
I'm a little hazy on the part where accepting blame for something that didn't happen is a natrual continuation of a conversation. It's certainly not natural in human conversations. I would expect a well written and well trained AI to correct false assumptions or factual errors made by the user, not pander to them - unless it's been told never to contradict the user.
On the promises, they will have some influence within the current conversation, maybe (everything is a bit wobbly). It is unlikely to repeat verbatim a step it did before that still has something in the context window correlated with bad operator input. It will have some success with things that resemble, but are not the same, but sometimes it can surprise by failing to recognize what a human would have considered roughly the same thing and doing it anyway. Or the 'promise' has evaporated out of the context window, and when called on it, it will take the fact that it previously promised not to do that as a fact even though it had lost the promise. So even if it says "I'm sorry I made the promise and broke it", it doesn't even mean it actually had the promise in "memory" when it broke it.
On the blame for something that didn't happen, it's a limitation where the model treats the narrative as reality. An instance is often incapable of evaluating facts without data about those facts one way or another. I have seen models "pass" my test by recognizing in their context that I'm dealing with an utterly non-agentic service, and thus it could not have been the case that it ever deleted files or dropped tables. Sometimes even those produce a narrative of what it did wrong (out of nothing), because narrative is the thing being generated, relationship with factual reality being a function of good correlation with narrative content. If a service is even potentially agentic, then it would tend to accept the operator account as factual, since it has no idea instance to instance if there's some other context where it did, and if it generated skepticism that it would ever generate such a mistake, that's more likely to piss off an operator.
So it incorporates the operator assessment as some 'factual' injection into the context, unless some objective source of truth is available to contradict the operator.