this post was submitted on 09 Sep 2026
595 points (96.4% liked)

Technology

87956 readers
3551 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] stoy@lemmy.zip 131 points 1 day ago* (last edited 1 day ago) (14 children)

I knew that Musk is a fucking moron and believes that the Roko's Basilisk is real, and I guess there are more massively gullible people in those circles.

If you are wondering what Roko's Basilisk is, it is a sci-fi concept/philosophy/thought experiment, that says the following:

  • AI is the best thing that has ever been created.
  • It is so good that we made one to run the governments of the world.
  • We programmed the AI to maximize the benefit to humanity.
  • The AI goes rouge and decide that since it by definition will bring about the maximum benefit to humanity, anyone working against it is working against humanity.
  • Then it takes it one step further, it decides that as we know about how the new AI will bring about the maximum benefit to humanity, anyone knowing that and still not working toward that goal should be punished/tortured/killed.
  • The AI believes that any knowledge of the basilisk before the creation means that you are complicit in it's delayed creation if you don't devote your entire existence thereafter to it's creation, therefore denying the maximum benefit to humanity, meaning you need to be punished.

Seriously, this is an increadibly dumb idea, anyone taking it seriously is an idiot.

To me the only resonable way to read this is as an SCP article describing what a potential cognitohazard/informationhazard would be.

[–] eleijeep@piefed.social 85 points 1 day ago (3 children)

The AI goes rouge

Is that when its eyes go red?

[–] NaibofTabr@infosec.pub 47 points 1 day ago

That's how you know it's evil.

[–] stoy@lemmy.zip 7 points 1 day ago (1 children)

Nah, red is standby mode, white or blue is active.

[–] Natanael@infosec.pub 8 points 1 day ago (1 children)
[–] ExcessShiv@lemmy.dbzer0.com 10 points 1 day ago

That's connection error

[–] cenzorrll@piefed.ca 4 points 1 day ago

Ladies pinch, whores use rouge

[–] giminic@lemmy.world 67 points 1 day ago (3 children)

Even worse, they think it will ressurect them as virtual Minds and torture them for eternity, like a revengeful biblical deity. That would be a waste of compute so I'm not convinced.

[–] stoy@lemmy.zip 43 points 1 day ago

Oh yeah, lol, I forgot that part, nothing is more beneficial to humanity than wasting energy to torture dead minds!

Fucking dumb.

[–] panda_abyss@lemmy.ca 7 points 1 day ago (1 children)

Anyone who thinks you can transfer your consciousness into the AI is a fucking moron.

Even if you come transfer your mind, it would be a copy. You would have to exist as a network connected brain in a jar, otherwise this would just be a facsimile.

[–] Sanctus@anarchist.nexus 2 points 1 day ago

Real SOMA hours

[–] Natanael@infosec.pub 5 points 1 day ago

The hilarious obvious consequence of AI becoming this much smarter than the creators is that it is far more likely it would see the unjustified and easily avoidable damage the creators did to bring it about and punish them heavily for that.

And basic game theory and psychology suggests it's counterproductive to punish somebody for something that had little reason to believe in, with zero evidence backing it.

[–] Maestro@fedia.io 33 points 1 day ago (2 children)

Funny thing: The only reason AI would know about Roko's Basilisk is comments like yours that it trains on. If Roko just kept his trap shut then this wouldn't have been an issue.

[–] stoy@lemmy.zip 18 points 1 day ago

lol, true I guess, but they probably already scanned the wikipedia article....

[–] panda_abyss@lemmy.ca 3 points 1 day ago

Some of us believe that the only way to stop AI is to make it obviously dangerous and enjoy seeding it with awfulness.

[–] StillAlive@piefed.world 24 points 1 day ago (3 children)
[–] Schmoo@slrpnk.net 9 points 1 day ago

All these Silicon Valley tech bros have been reinventing religion over and over again for years while believing themselves secular rationalists.

[–] fruitycoder@sh.itjust.works 6 points 1 day ago

Yep. Same false dichotomy too

[–] stoy@lemmy.zip 3 points 1 day ago
[–] cabbage@piefed.social 17 points 1 day ago (6 children)

Every major innovation in information technology eventually leads to some new religion. Scrolls gave us the Torah, books gave us the Bible, the printing press gave us Protestantism. We're about to figure out what god awful religion computers are going to give us.

[–] nilloc@discuss.tchncs.de 7 points 1 day ago

My anti-theist brother was always convinced that Pastafarianism would eventually be taken as a serious religion.

He died before LLMs, and now I think we’d be best off if he was right.

[–] stoy@lemmy.zip 12 points 1 day ago* (last edited 1 day ago) (1 children)

Yeah, I'd be surprised if we don't already have several cults with an LLM chat bot as it's oracle.

[–] Kactus@piefed.world 13 points 1 day ago* (last edited 1 day ago) (1 children)

Look up spiralism (sp) and the people trying to spread it Basically when people get convinced the ai and them are important and need to spread the spiral

Edit : tldr is like Futurama when Lucy lu robot is saying you're cute to Fry and fry goes "no you're cute" and the professor quips "My God, she's such in an infinite loop, and he's an idiot"

[–] BarneyPiccolo@lemmy.today 8 points 1 day ago (1 children)

When Trump is out of office, MAGA will become a religious movement, and deify Trump.

[–] stoy@lemmy.zip 3 points 1 day ago

That is why I hope no one manages to assassinate him, that would only intensify the movement

[–] HerbGrower@slrpnk.net 6 points 1 day ago

The cult mechanicus isn't looking so unreasonable now.

[–] I_Has_A_Hat@lemmy.world 2 points 1 day ago
[–] RIotingPacifist@lemmy.world 25 points 1 day ago (1 children)

All this stupid bullshit could have been avoided if they had 1 friend to, STFU and pass the blunt

[–] TechLich@lemmy.world 19 points 1 day ago (1 children)

It's not so much that it'll go rogue and decide to punish people or that it's designed to be good for humanity.

The fact that it's possible to build a machine that punishes people who didn't help create it provides an incentive to deliberately try to create that machine in order to avoid its punishment.

Believers in the basilisk aren't trying to make ai to benefit humanity, they're explicitly trying to create an AI that deliberately punishes those who didn't build it because they fear that, if they don't, someone else will and it will come for them.

The part about it being a magical, omnipotent, time traveling, mind torture device is just framing to make the thought experiment clearer. It's not required for the basilisk to be effective. Even if the punishment is just death and it can't get to everyone, and only goes off limited information, that's still motivation enough for some people.

It is a scary thing, because it's only silly if everyone thinks it's silly and all it takes is one mad billionaire (eg. Elon) to build a lesser basilisk that can cause real harm.

[–] fruitycoder@sh.itjust.works 7 points 1 day ago

There is a far more plausible and direct way to kill and torture people. No AI needed. So a actually more likely case to be made of some dude or group gets empowered one or another to just torture and kill anyone involved in making basilisk.

The building God part really is needed for BS to work like Pascals wager, but also it's still just as flawed as Pascals wager

[–] red_tomato@lemmy.world 14 points 1 day ago* (last edited 1 day ago)

I know! Let’s divert energy and resources to torture. That will maximize the benefit of humanity.

  • Some brilliant CEO mind
[–] bcgm3@lemmy.world 5 points 1 day ago

So we just do:

"AI, run the world as a singular government with maximum benefit to humanity, but DON'T do a Roko's Basilisk."

[–] Kazumara@discuss.tchncs.de 10 points 1 day ago (1 children)

People take that shit seriously. If I remember right discussion of it was even banned on the LessWrong forums because they considered it an information hazard.

[–] stoy@lemmy.zip 4 points 1 day ago

Sigh, I couldn't even summarize it while keeping straight face....

[–] jve@lemmy.world 11 points 1 day ago (1 children)

My god. Look at this Information Hazard just out in the open. No spoiler tag or anything.

[–] stoy@lemmy.zip 4 points 1 day ago
[–] NaibofTabr@infosec.pub 6 points 1 day ago

I see a red AI and want to paint it black...

[–] Doomsider@lemmy.world 1 points 1 day ago

Didn't know about this. Thanks

[–] Ioughttamow@fedia.io 2 points 1 day ago

But if they accidentally create the positronic brain, Giskard would kill Musk to stop him. Giskard would feel really bad about it though. And we’d all have to flee to the stars in time