Technology

I disagree with the "Pandora's box is open" angle because my beef isn't with the technology, but how it's being used in practice. It's a socioeconomic problem, but a technological one.

Cory Doctorow articulates it much better than I can^[1]:

"Now, if AI could do your job, this would still be a problem. We'd have to figure out what to do with all these technologically unemployed people.

But AI can't do your job. It can help you do your job, but that doesn't mean it's going to save anyone money. Take radiology: there's some evidence that AIs can sometimes identify solid-mass tumors that some radiologists miss, and look, I've got cancer. Thankfully, it's very treatable, but I've got an interest in radiology being as reliable and accurate as possible.

If my Kaiser hospital bought some AI radiology tools and told its radiologists: "Hey folks, here's the deal. Today, you're processing about 100 x-rays per day. From now on, we're going to get an instantaneous second opinion from the AI, and if the AI thinks you've missed a tumor, we want you to go back and have another look, even if that means you're only processing 98 x-rays per day. That's fine, we just care about finding all those tumors."

If that's what they said, I'd be delighted. But no one is investing hundreds of billions in AI companies because they think AI will make radiology more expensive, not even if that also makes radiology more accurate. The market's bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: "Look, you fire 9/10s of your radiologists, saving $20m/year, you give us $10m/year, and you net $10m/year, and the remaining radiologists' job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it's catastrophically wrong.

"And if the AI misses a tumor, this will be the human radiologist's fault, because they are the 'human in the loop.' It's their signature on the diagnosis."

This is a reverse centaur, and it's a specific kind of reverse-centaur: it's what Dan Davies calls an "accountability sink." The radiologist's job isn't really to oversee the AI's work, it's to take the blame for the AI's mistakes."

Even with the technological limitations that AI faces at the moment, we could be doing so much more with it. I love this radiography example because so many of us have experienced someone in our life getting cancer. AI is absolutely capable of improving the rate at which we are detecting cancer at an early stage, which would absolutely save lives. Instead what we're getting is that it is being used as an excuse to heap more work onto doctors and radiographers, worsening the situation for everyone.

I do agree with the broad strokes of what you're saying, because absolutely it does take time for any new technology to integrate itself into society and become useful. However, I don't believe that AI in its current form is capable of becoming commercially viable (and by "in its current form", I am talking about a paradigm that demands excessive building of super resource intensive datacentres)

Edit: forgot to add the citation [1]: https://pluralistic.net/2025/12/05/pop-that-bubble/