Technology

Good job in helping with their guerilla marketing campaign.

When a corporation raises that working for government will be more profitable than working for consumers, they pivot to scaremongering, but always with the undercurrent of "and that's why you should PAY US so that you own the nukes, and not your enemies"

Personally, this just feels like marketing and these big asshole companies trying to resurrect the 2025 level of hype in order to salvage their IPO's.

If they honestly believed this I can't imagine that their workers wouldn't do something more drastic.

In addition to what everyone else said, I'll tell you why I don't think that what we have currently can become some runaway superhuman intelligence:

All “AI” people are talking about are basically “just” LLM wrappers: tools built to run a LLM in a loop while giving it access to the ability to write and run code. This gets pretty powerful, since you can do a lot programmatically (including monetary transactions), but has a fundamental limit:

LLMs are trained ahead of time. And that's a very cost intensive, long process. Beyond that, the LLM has only a context window of finite length. So there are no long-term planning capabilities. While running after being trained, the LLM can only condense knowledge into a summary and then re-read it, a lossy process that might introduce mistakes in “understanding” and will make any prolonged task degrade over time.

And even if they figure out how to make the things actually self-modify and learn things, you can't give every session that ability, since that'll vastly increase the storage and performance requirements. Do sessions with that ability will be closely monitored and limited.

So I don't see any space where an evil AI could emerge, even when factoring in an essential possible future improvement to the technology.

The people claiming it’s humanity ending are all directly involved in the AI companies. One take is that they are the ones that know the most about it.

My take is they are financially incentivized in making sure their investments grow in value.

Humanity should be way more frightened of greedy psychopaths running the world than any technology.

I knew a guy in high school that told girls they shouldn't date him because his dick was so big it would hurt them. You can wax philosophical on his motives if you'd like, but spoiler alert - his dick wasn't big.

The biggest risk I see is Idiocracy. Like, the movie.

Business workers become afraid to do or propose anything without AI signoff. Retail & service workers are told to do what the AI says, the way the AI says to do it, or get fired.

Technology innovator Marshall Brain (RIP) wrote a book in the early 2000s about an AI called "Manna" designed to replace fast food managers:

https://web.archive.org/web/20201204080050/https://marshallbrain.com/manna1

The AI just talks in your ear all day: "Empty one bag of fries into the fryer. You have 6.5 minutes, switch to the sandwich station. Make one Big Burger: grab one patty from the 1/4 pound bin and place it on the griddle... ... The fries are ready, turn around and lift them into the drain hopper..."

All day, every day, exactly 8 hours plus mandatory breaks, employees on autopilot listening on ear buds.

I'm more worried about the people using AI than the tech itself.

Ed Zitron’s podcast “Better Offline” recently talked about this. There is a small tech community called the Rationalists who believe AI super intelligence is inevitable and it will either accelerate humanity to new heights or kill off the species. These rationalists are actually a very prominent faction within tech and the AI industry. Apparently the language used within top AI firms is largely about the immense danger posed by AI super intelligence if any other group or company were to achieve it first. This creates an echo chamber that makes the people in the industry kind of lose their mind and misinterpret the capability of LLMs.

Everyone outside that faction recognizes that LLMs are just unreliable cloud software with limited use cases and that there’s no evidence LLMs could actually lead to any kind of intelligence.

Edit: Please, someone stop me from commenting on an AI post next time I’m tempted. I’m so tired of people replying, arguing with opinions I don’t have, based entirely around a hypothetical technology that doesn’t exist yet. LLMs make slop. Thats it. They hacked a company after weeks of 100s of instances being run, using old exploits published online. San Francisco will be underwater from climate change before LLMs can do anything more.

Whatever. Ban all these people from employment and public assistance. Make them starve

Look up “rationalists” and “zizians”

It will quickly become clear to you that the dichotomy is caused by one side being completely off their fucking rockers.

It's been reported a few times that Ukraine is already using AI for drones.

The human operator chooses the target, the AI takes over and eliminates the target.

Is that "just an LLM"?

Seems more capable than that to me. But, I don't know if this is actually true, or just a claim to frighten the Russians (and me).

I'm not pro-AI by any stretch for a huge number of reasons, but in my day job as a technical product manager I have to use a number of AI-powered tools and workflows. Whereas 18 months ago they were a novelty that the bosses were enamored of, today they get real work done and speed up a number of key areas, mainly like fancy automation workflows (which have been around forever). The rate of improvement and capability for certain use cases has been very, very rapid. We do seem to be reaching a plateau though. The tools and models from 6 months ago are just about as good as the ones from today. And China's models from today are better than the US's models from 6 months ago, but at a fraction of the cost.

So when I hear people talking about the "end of the world", I think a realistic case for that is that the AI tools that come out in the next few years will be good enough to do a fair amount of white-collar work, and we won't have a plan in place to deal with all of the now out-of-work people that leads to. Whole classes of jobs in areas like data entry, telemarketing, and customer support will be almost entirely automated out of existence. New classes of jobs will be created, of course, but it's hard to imagine them existing in sufficient numbers to make up for the losses. And this is just for LLM-workloads.

In other areas, other types of AI are already enabling robotic automation of factories and in the near-ish future might make self-driving truck fleets practical. This would be devastating because factory workers and truck drivers make up like 30% of the US workforce -- and again, there are no signs of new classes of jobs that could absorb the out-of-work workers, and no plans for a social safety net or wealth distribution.

So I'm not worried about a rogue AI unleashing some bioweapon or launching a nuclear attack. I'm worried about a bunch of CEOs deploying barely-acceptable AI to displace millions of workers as soon as it is safe(ish) to do so.