Technology

I disagree with the "Pandora's box is open" angle because my beef isn't with the technology, but how it's being used in practice. It's a socioeconomic problem, but a technological one.

Cory Doctorow articulates it much better than I can^[1]:

"Now, if AI could do your job, this would still be a problem. We'd have to figure out what to do with all these technologically unemployed people.

But AI can't do your job. It can help you do your job, but that doesn't mean it's going to save anyone money. Take radiology: there's some evidence that AIs can sometimes identify solid-mass tumors that some radiologists miss, and look, I've got cancer. Thankfully, it's very treatable, but I've got an interest in radiology being as reliable and accurate as possible.

If my Kaiser hospital bought some AI radiology tools and told its radiologists: "Hey folks, here's the deal. Today, you're processing about 100 x-rays per day. From now on, we're going to get an instantaneous second opinion from the AI, and if the AI thinks you've missed a tumor, we want you to go back and have another look, even if that means you're only processing 98 x-rays per day. That's fine, we just care about finding all those tumors."

If that's what they said, I'd be delighted. But no one is investing hundreds of billions in AI companies because they think AI will make radiology more expensive, not even if that also makes radiology more accurate. The market's bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: "Look, you fire 9/10s of your radiologists, saving $20m/year, you give us $10m/year, and you net $10m/year, and the remaining radiologists' job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it's catastrophically wrong.

"And if the AI misses a tumor, this will be the human radiologist's fault, because they are the 'human in the loop.' It's their signature on the diagnosis."

This is a reverse centaur, and it's a specific kind of reverse-centaur: it's what Dan Davies calls an "accountability sink." The radiologist's job isn't really to oversee the AI's work, it's to take the blame for the AI's mistakes."

Even with the technological limitations that AI faces at the moment, we could be doing so much more with it. I love this radiography example because so many of us have experienced someone in our life getting cancer. AI is absolutely capable of improving the rate at which we are detecting cancer at an early stage, which would absolutely save lives. Instead what we're getting is that it is being used as an excuse to heap more work onto doctors and radiographers, worsening the situation for everyone.

I do agree with the broad strokes of what you're saying, because absolutely it does take time for any new technology to integrate itself into society and become useful. However, I don't believe that AI in its current form is capable of becoming commercially viable (and by "in its current form", I am talking about a paradigm that demands excessive building of super resource intensive datacentres)

Edit: forgot to add the citation [1]: https://pluralistic.net/2025/12/05/pop-that-bubble/

I think the primary gripe with AI is less about having no potential, but about the specific potential for evil being leveraged heavily while "visionaries" lie about or dismiss the (current and foreseeable future) limits for clout and profit.

To use your comparison, CEOs selling solar panels are responding to scepticism about the stability of that supply and its elasticity to fluctuating power demand with vague promises of developing more advanced cells that can operate with less light and throttle the generated power during ebbs in demand. They buy land to install more solar panels, then lobby for the deactivation of other, flexible power sources that could supplement their supply until they can finally start charging more in times of high demand.

They do so because the alternative would be admitting that the product they're selling isn't ready for the market they're trying to enter (or at least not at the scale they're aiming for) and that the solution to their issues requires different technologies which have not yet been developed. They can't admit that because it would upset the investors that trusted their vision. As a famous adage suggests, they probably can't even understand that limit, since their salary depends on not understanding it.

In the same vein, the current CEOs fail (or refuse?) to acknowledge that the generative models today have a fundamental limitation with understanding the semantics behind certain patterns. One infamous phenomenon would be AI hands: We know what a hand is and how it works; a generator can only vaguely imitate the patterns. It doesn't know "hand" or "finger", so it just generates some assembly that vaguely fits its training material.

If we consider the broader and ill-defined field of AI, I can believe that some form of semantic modelling technology could provide a way to connect tokens with abstract concepts and enable a type of reasoning that pure token predictors are incapable of. But we first need to acknowledge that limit, and on that front, we're competing against tech-bros that would rather believe than know and against grifters that would rather sell snake oil than teach medicine.

As much as I also just think AI is just going to make lazy people lazier, and we’ve already seen some effects regarding our ability to problem solve, the real issue is that none of the wonderful utopian AI stuff is going to come true at the rate we’re going. I don’t guve a fat flying fuck how good or bad AI is, I care that it’s being used to spy on us, make decisions no human will ever review, and has already led to mass layoffs without any kind of safety net for those people. I care that it’s raping the environment so some sentient dick-cheese can get worse results than a near-zero effort search would give them. Increased productivity has yet to give anyone more free time, they just axe someone else and overwork whoever’s left.

We are NOT ready for AI at ANY level. It’s a weapon being used against the working class. If you think the issue is with its quality then you need to sit the fuck down, shut the fuck up, and pay some goddamn attention.