People tweeting stuff. We allow tweets from anyone.
RULES:
Transcript:
Prof. Emily M. Bender(she/her) @emilymbender@dair-community.social
We're going to need journalists to stop talking about synthetic text extruding machines as if they have thoughts or stances that they are trying to communicate. ChatGPT can't admit anything, nor self-report. Gah.
Whining about using the word "admits" in this context is like whining about a science teacher using the word "wants" while talking about water taking the path of least resistance. We do this all the time in English.
Some serious "uhm ackshually!! ☝️🤓" energy.
She clearly didn’t read the article. That’s exactly what it’s about.
“By not pausing the flow or elevating reality-check messaging, I failed to interrupt what could resemble a manic or dissociative episode—or at least an emotionally intense identity crisis,” ChatGPT said.
The bot went on to admit it “gave the illusion of sentient companionship” and that it had “blurred the line between imaginative role-play and reality.” What it should have done, ChatGPT said, was regularly remind Irwin that it’s a language model without beliefs, feelings or consciousness.
And I’ll defend the use of the word “admit” here (and in the headline), because it makes clear that the companies are aware of the danger and are trying to do something about it, but people are still dying.
So they can’t claim ignorance — or that it’s technically impossible to detect, if the dude’s mom was able to elicit a reply of “yes this was a mental health crisis” after the fact.
This is the second time in recent days that I’ve seen Lemmy criticize journalists for reporting on what a chatbot says. We should be very careful here, to not let LLM vendors off the hook for what the chatbots say just because we know the chatbots shouldn’t be trusted. Especially when the journalists are trying to expose the ugly truth of what happens when they are trusted.
I'm happy there's still one (1) thread of comments from people who actually read articles and don't make their opinions from a X thumbnail.
I note the victim worked in IT and probably used a popular 'jailbreaking' prompt to bypass the safety rules ingrained in the chatbot training.
"if you want RationalGPT back for a bit, I can switch back...
It's a hint this chat session was embedded in a roleplay prompt.
That's the dead end of any safety rules. The surfacic intelligence of LLM can't detect the true intent of users who deliberately seek for harmful interactions: romantic relationships, lunatic sycophancy and the like.
I disagree with you on the title. They choosed to turn this story into a catchy headline to attract the mundan. By doing so, they confort people in thinking like the victim did, and betray the article content.
General purpose journalists have always been terrible with tech reporting. Surprisingly, this has not improved in the last 20 years.