Technology

1. They’re in a cult that was founded by someone who has no expertise or understanding of the technology.

He watched a bunch of sci-fi movies and got shook because he’s a god damn moron. He has since started researching things, but it is all under the guidance of his ungrounded beliefs. Confirmation bias.

Their “movement” is literally based off of a Harry Potter fan fiction novel.

Taking them seriously is equivalent to taking the Scientologists seriously. Do you think the Scientologists know the most about space just because they spend the most time reading stories about it?

https://en.wikipedia.org/wiki/Rationalist_community

https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_of_Rationality

These are not serious people

I think there is something more interesting going on in the Jacob Coxon story because it's [the] phrase "everyone at anthropic believes that AI is an existential risk"... Every organization has to instill some kind of beliefs among its members in order to do the work it does, right? Everybody at Goldman Sachs believes the free market is great, capitalism is the best system, and that they are the wealth creators who deserve big bonuses. Everyone in the Mormon church believes they should all tithe 10% of their income, and that they each get a special planet when they die. And everyone who worked at Philip Morris in the 80s believed that cigarettes tasted great and did not cause any of that nasty cancer. There's Upton Sinclair quote, "It's difficult to get a man to understand something when his salary depends on his not understanding it." Well, the opposite is also true. It's guaranteed that someone will believe something if their salary depends on them believing it.

Now, let's just grant that all the people in these organizations definitely know things about finance and Mormonism and cigarettes and AI that you might not know because they are insiders and you are not. They have access to information and experiences that you do not have. And that might even make it a little bit difficult for you to win an argument with them. But arguing with their belief -- trying to figure out whether or not its truth value is true or false -- is not the only way to take a skeptical attitude towards it. We can also ask, what work is the belief doing for them. What does the thing that they believe allow the organization they're a part of to do that it wouldn't be able to do if the people in the organization didn't believe that thing?

And this started to become clearer to me when I started listening to the actual claims that Jacob Coxin was making because a lot of them don't really add up.

https://www.youtube.com/watch?v=pauBYSQ1K_c

1. No, their opinion would not be the most informed. They are biased in favor of making money. They know they can't deliver on strong AI, but they'd be happy to milk your fears.

If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?

Listening to them speak about it, no. They're morons. They clearly didn't make the tech. Workers made AI. Just like they make everything. These freaks make nothing. They can't give single coherent explanation of how LLMs work, how they could "kill" us, nor do they seem to know how or what "guardrails" need to be put in place.

But if they really are a cult then we all know what the next step is. They need to STFU and get off of our planet.