This is a most excellent place for technology news and articles.
For reference, this is roughly equivalent to playing a PS5 game for 4 minutes (based on their estimate) to 10 minutes (their upper bound)
calulation
source https://www.ecoenergygeek.com/ps5-power-consumption/
Typical PS5 usage: 200 W
TV: 27 W - 134 W → call it 60 W
URI's estimate: 18 Wh / 260 W → 4 minutes
URI's upper bound: 48 Wh / 260 W →10 minutes
That are 25 request per kWh. At 10 to 25cents per kWh that's 1cent per request. That doesn't seem to be too expensive.
And an LLM that you could run local on a flash drive will do most of what it can do.
I mean no not at all, but local LLMs are a less energy reckless way to use AI
Probably not a flash drive but you can get decent mileage out of 7b models that run on any old laptop for tasks like text generation, shortening or summarizing.
What do you use your usb drive llm for?
Porn. Obviously.
Fucking Doc Brown could power a goddamn time machine with this many jiggawatts, fuck I hate being stuck in this timeline.
Help me out here. What designates the “response” type? Someone asking it to make a picture? Write a 20 page paper? Code a small app?
Response Type is decided by ChatGPTs new routing function based on your input. So yeah. Asking it to "think long and hard", which I have seen people advocating for to get better results recently, will trigger the thinking model and waste more resources.
So instead of just saying "thank you" I now have to say "think long and hard about how much this means to me"?
If you want it to really use a lot of energy on receiving your gratitude, sure I guess^^
I have an extreme dislike for OpenAI, Altman, and people like him, but the reasoning behind this article is just stuff some guy has pulled from his backside. There's no facts here, it's just "I believe XYX" with nothing to back it up.
We don't need to make up nonsense about the LLM bubble. There's plenty of valid enough criticisms as is.
By circulating a dumb figure like this, all you're doing is granting OpenAI the power to come out and say "actually, it only uses X amount of power. We're so great!", where X is a figure that on its own would seem bad, but compared to this inflated figure sounds great. Don't hand these shitty companies a marketing win.
Thats actyally a fav rhetorical trick of mine when arhuing with consummatw bullshitters who have followers.
I don't care how rough the estimate is, LLMs are using insane amounts of power, and the message I'm getting here is that the newest incarnation uses even more.
BTW a lot of it seems to be just inefficient coding as Deepseek has shown.
For training yes, but during operation by this studies measure Deepseek actually has an even higher power draw, according to the article. Even models with more efficient programming use insane amounts of electricity
This was higher than all other tested models, except for OpenAI's o3 (25.35 Wh) and Deepseek's R1 (20.90 Wh).
OK I guess I didn't read far enough but your quote says that Deepseek uses less than Open AI?
And water usage which will also increase as fires increase and people have trouble getting access to clean water
https://techhq.com/news/ai-water-footprint-suggests-that-large-language-models-are-thirsty/
It would only take one regulation to fix that:
Datacenters that use liquid cooling must use closed loop systems.
The reason they dont, and why they setup in the desert, is because water is incredibly cheap and energy to cool a closed loop system is expensive. So they use evaporative open loop systems.
Closed loop systems require a large heat sync, like a cold water lake, limiting them to locations that are not as tax advantageous as dry red states.
Unfortunately I wonder if it’s more expensive to set up a closed loop system that’s really expensive or to buy lawmakers that will vote against bills saying you should do so and it’s a tale old as time
BTW a lot of it seems to be just inefficient coding as Deepseek has shown.
Kind of? Inefficient coding is definitely a part of it. But a large part is also just the iterative nature of how these algorithms operate. We might be able to improve that via code optimization a little bit. But without radically changing how these engines operates it won't make a big difference.
The scope of the data being used and trained on is probably a bigger issue. Which is why there's been a push by some to move from LLMs to SLMs. We don't need the model to be cluttered with information on geology, ancient history, cooking, software development, sports trivia, etc if it's only going to be used for looking up stuff on music and musicians.
But either way, there's a big 'diminishing returns' factor to this right now that isn't being appreciated. Typical human nature: give me that tiny boost in performance regardless of the cost, because I don't have to deal with. It's the same short-sighted shit that got us into this looming environmental crisis.
I think AI power usage has an upside. No amount of hype can pay the light bill.
AI is either going to be the most valuable tech in history, or it's going to be a giant pile of ash that used to be VC capital.
that's a lot. remember to add "-noai" to your google searches.
This is my weekly time to tell lemmings about Kagi, the search engine that does not shove LLM in your face (but still let's you use it when you explicitly want it) and that you pay for with your money, not your data.
The team measured GPT-5’s power consumption by combining two key factors: how long the model took to respond to a given request, and the estimated average power draw of the hardware [they believe is] running it.
Bit of a clickbait. We can't really say it without more info.
But it's important to point out that the lab's test methodology is far from ideal.
The team measured GPT-5’s power consumption by combining two key factors: how long the model took to respond to a given request, and the estimated average power draw of the hardware running it.
What we do know is that the price went down. So this could be a strong indication the model is, in fact, more energy efficient. At least a stronger indicator than response time.
