this post was submitted on 27 Mar 2026
309 points (97.0% liked)

Technology

83158 readers
4223 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
309
Huge Study of Chats Between Delusional Users and AI Finds Alarming Patterns (futurism.com)
submitted 1 day ago by return2ozma@lemmy.world to c/technology@lemmy.world
89 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] givesomefucks@lemmy.world 42 points 1 day ago (2 children)

As the researchers wrote in a summary of their findings, the “most common sycophantic code” they identified was the propensity for chatbots to rephrase and extrapolate “something the user said to validate and affirm them, while telling them they are unique and that their thoughts or actions have grand implications.”

There's a certain irony in all the alright techbros really just wanting to be told they were "stunning and brave" this whole time.

[–] A_norny_mousse@piefed.zip 3 points 22 hours ago

Huh. I hate it when people do that. Fake/professional empathy/support. Yet others gobble it up when a machine does that.

[–] Tiresia@slrpnk.net -3 points 17 hours ago (1 children)

Are the users in this study techbros?

Besides, tech bros didn't program this in, this is just an LLM getting stuck in the data patterns stolen from toxic self-help literature.

For decades there has been a large self-help subculture who consume massive amounts of vacuous positive affirmation produced by humans. Now those vacuous affirmations are copied by the text copying machine with the same result and it's treated as shocking.

[–] very_well_lost@lemmy.world 2 points 7 hours ago

Besides, tech bros didn't program this in, this is just an LLM getting stuck in the data patterns stolen from toxic self-help literature.

That's not necessarily true. The AI's output is obviously shaped by the training data, but much of it is also shaped by the prompt (and I don't just mean your prompt as a user).

When you interact with (for example) ChatGPT, your prompt gets merged into a much larger meta-prompt that you don't get to see. This meta-prompt includes things like what tone the AI should use, how the AI should identify itself, how the AI should steer the conversation, what topics the AI should avoid, etc. All of that is under the control of the people designing these systems, and it's trivially easy for them to adjust the way the AI behaves in order to, for example, maximize your engagement as a user.