Fuck AI

Through no choice of our own

• "The web" is an open protocol aimed for interoperability

Yes, although the reality today is that you have to spend a decent amount of money to be on the web effectively. You could run a webpage on an RPi on your home network, but that won't do for much more than a few visitors a day, and it involves several compromises and some security risks.

There isn't really an equivalent to the web's interoperability with these AI systems, except maybe if you're using it to control smart home devices.

• "The AI" is a bunch of competing, mostly closed and private services

Yeah, although being able to run OpenClaw &etc locally is interesting. Most people won't ever be able to train their own models, but at least self-hosting some of them is an option.

The cost per token goes up with every new generation of models. It goes down over time as the model undergoes optimizations such as distillation. Unfortunately, those optimizations make the model perform worse and more likely to hallucinate. Then they come out with a new generation which obviously outperforms the heavily watered down, distilled model, and the cycle repeats again.

Think of a trend like this:

The costs keep going down but the long term trend is going up.

The cost is not actually going down. The price is going down, because these companies are need new revenue and new customers and new waves of hype to prop up their failing business model. The cost is at best staying level if you want to be very selective and generous, and is inevitably going up. And it's going to go up much, much faster in the near future.

Sure, Nvidia may put out some newer chips that are technically somewhat more efficient from time to time, but the old cards are still running, they've already been purchased and they still have to pay for themselves, and there is no evidence they have (or ever will) and the new ones are even more expensive than that, and they'll have to pay for themselves too. And at the same time energy is getting more expensive, training costs are getting more expensive, and demand on both sides is increasing which is only going to push those costs even higher.

There is zero possibility any of this makes economic sense. The entire economic output of the world decided to jump on the FOMO bandwagon, that doesn't mean the wagon is actually going anywhere.

I'm not certain, but it could be that as the cost goes down, they keep bringing out more powerful models, so the cost per query stays constant.

If this is accurate, their need to compete with each other to have "the best" AI is creating a prisoner's dilemma where everyone loses because the individually-optimal choice is worse than if they all agreed that today's AI is good enough and concentrated on affordability and minimising energy/water usage instead.

They’re heavily subsidizing the costs to gain users who otherwise probably won’t be interested in the service at a sustainable cost. Every company is hiding their inference costs, but it’s clear that every user is currently burning far more than they’re generating in revenue. The hope is that inference costs will go down, and while that’s a fairly safe bet, there’s two problems:

1. Frontier model companies are burning cash so fast, they’ll run out long before economies of scale will make the costs affordable.
2. Even if the per-token inference costs have gone down, almost every technique (thinking, large context windows, etc) to improve AI performance has involved increasing the number of tokens used. Total query cost is easily outpacing any decrease in per-token inference cost.

Even worse, models themselves are becoming commodities. Although users seem to have preferences for one model over others, there’s still not really a good way to benchmark them. Without a clear ability to differentiate models on performance or ability they’re completely interchangeable, which lowers margins. Why pay more to run company X’s latest and greatest, when company Y’s last generation performs almost identically?

The reason the web was able to cover costs with advertising is because the cost to serve a web page was minimal. A bit of networking gear and a couple servers was all you needed to serve a large website. For many sites, you didn’t even need premium hardware, just a cheap, basic PC with an Internet connection. Lots of people ran free hobby websites with minimal cost. Hell, you can run a website on a single board computer like a Raspberry Pi.

By contrast, AI needs huge GPU clusters to respond to a prompt. A four year old H100 GPU will cost around $30,000; typically 8 of those are clustered together in systems that cost more than $300,000. I can’t even find costs for current generation B100 GPUs or B200 clusters, only cloud rentals. Serving an AI model is orders of magnitude more expensive than serving a website.

The problem is how much computation is required to handle every user request.

When the Internet was starting out, most websites weren't much more than text, maybe some low-resolution pictures. Even in the '90s, serving that content to users was computationally cheap. A company's web server could just be a desktop in the basement.

AI models are expensive to train and expensive to operate. Just maintaining the environmental needs for the massive data centers is a significant cost. Charging users for access is not nearly enough to cover the expenses, by orders of magnitude, and they're already in massive debt.

Yes, YouTube is a great example of a site that burned shitloads of cash in exactly the manner described until Google figured out a monetization strategy that worked.

Sure but my post, while not fully committing to any particular analysis of the future prospects of the sustainability or future success of the current AI technologies that are being discussed here, was about taking the idea that they're all hype and will result in a burst bubble as true for the purposes of discussion and wondering if that will have negative consequences for everyone else like me, who has nothing to do with any of it. My point was I'm usually glad in a schadenfreude way, to hear about people thinking it's a bubble that will burst because it sounds like a stay of execution for me and most of society but I'm wondering if, despite never wanting it to succeed, the failure of AI is going to end up having consequences for everyone or if it will be contained just to those working directly in it or who are heavily invested in it. I'm not really weighing-in on the veracity of the claims of a bubble just looking for perspective on what happens if it IS one.