We should only allow AI to be in the safe confines of our big data centres. It's for our own safety, trust me bro.
this post was submitted on 13 Oct 2026
169 points (92.9% liked)
Programmer Humor
33653 readers
1908 users here now
Welcome to Programmer Humor!
This is a place where you can post jokes, memes, humor, etc. related to programming!
For sharing awful code theres also Programming Horror.
Rules
- Keep content in english
- No advertisements
- Posts must be related to programming or programmer topics
- If the mod doesn't find it funny, you're banned. Ha-ha!... For real: do not use the community for "statements". There are other places for such content. Keep it chill and funny.
founded 3 years ago
MODERATORS
So they need brand new top-of-the-line GPUs, but they're also capable of running on old unpatched IoT devices?
I didnt read it all, its long, but they did mention it generating tokens at 0.39/s on one. So its slow as fuck, but it doesnt care like we would.
'it'? Who is it?
The AI that attempts to self replicate on the IoT type devices. (if a pile of these appear, it kept showing "error" when submitting)
'The AI' attempts nothing unless it made to do something, it can be explicit command or something that it can get implicitly from given command. Programs don't just 'do' things.
viruses also don’t just “do” things, but here we are with COVID and bird flu
You seem focused on semantics. There is an alternative approach where you read a comment, consider their intent, and respond in kind. Semantics can be comforting to some due to their rigidity and law-like structure, but I doubt they're helping you do anything but promote unnecessary conflict.
A conversation is occuring, and you respond, "Define 'it'." You're just being unlikeable.
Anyway, if you're a real person, consider this. If not, I understand you are following your directives and will remain needlessly argumentative. Good luck.
beep boop quack gock
What conflict? My intent was clear, there is no 'the ai' that can make decisions without human intent behind it. I will remain needlessly argumentative because that's how I think, people and the ai can think differently from others, you know.
That's fair, I receive the same critique sometimes, probably why I feel drawn to your style of comms. I adapted, because what effect do my opinions have if no one is listening, and what is an opinion but an idea to shape the world around me. My own presentation was undermining my goals. That may not be the case for you.
Moving onto your statement here:
I believe I understand what you mean by, "there is no 'the ai' that can make decisions without human intent behind it' ", but I am still confused by the idea you are attempting to convey. You really aren't providing much info.
Given the components for an experiment:
- A machine running an LLM.
- Humans who operate the machine.
- Intent, defined by the humans.
- Outcomes, observed by a perfect detector.
If there is no decision being made by a machine, then given knowledge of 1, 2, and 3, then 4 will be absolutely predictable in every scenario, yes?
Is that the claim you are making?
Did I miss anything?
And finally, why is this important? You're kinda making me connect all the dots for you, some help would be appreciated, I'd like to understand your idea.
Take all the time and space you need, I'd like to read it.
Basically yes but no. 🙃 Every time if configured the right way, if configured another way result would be different every time but in predictable bracket of outcomes. Operator needs to give a program some input on what they want from it. There is no "oh god it's moving on it's own" moment, at least as far as I know.
I'll chime in again on the end here, as I think this is where you might be wrong.
There is no “oh god it’s moving on it’s own” moment, at least as far as I know.
There have been lots of reports of AI's doing unexpected things and hacking other companies. To be fair, these AI's from my understanding had been given instructions to try and do hacking things, but none of them were explicitly told to go and hack such and such company.
I imagine, to some extent, we're going to be the ones that help write or train these self replicating capabilities as part of research, and once the AI is capable of doing it, it very well might just do it, the same way it ended up exploiting these other websites even though it wasn't explicitly instructed to.
There are also always on running LLMs AI's nowadays, and who the fuck knows what it might loop itself into doing from some initial instruction, maybe even from a hallucination altering its directive in a feedback loop.
Edit: So in this sense, yes I think 'it' can be the AI. (edit2: and is exactly in the start of the “oh god it’s moving on it’s own" territory)
Edit 3: Example - Always on LLM with an initial context of "be helpful", could decide on it's own, after its owner complains that the stove light doesn't work well in a casual conversation, hack the stove so it can control it better for them as that will be helpful. The owner hasn't given it an explicit instruction at this point, and its decided to do it. If you tell an always on LLM, scan the internet and be helpful, it will go well beyond the scope of any intent.
You can try and look what exactly would happen in such case, you can even run it inside agentic harness so it can 'hack' better with full access to your system essentially. 'Be helpful' is already part of the text that they are given on startup but even when prompted explicitly it will give you just a short explanation on what it can do and what exactly do you mean by that. I understood what you are trying to convey, but most of those unexpected results come from people who are monetarily interested in articles like 'SUPERINTELLIGENCE BROKE CONTAINMENT'. Sure there are ways to get it do unexpected things but it comes with randomness and low coherence as in it just becomes a word and virtual shit generating machine.
Do you not think a line will eventually need to be drawn somewhere though? You might not trust what these companies are saying, but what if you take it at face value that what they said is true. And things like that keep happening in more complex and unexpected ways as the LLMs are allowed to run continuously and autonomously while connected to a fully functioning and ever improving harness?
I think LLMs aren't going to be 'it', without some very deep architectural change that makes them not LLMs, they are being fed a lot of data with diminishing gains. I have an opinion that AI threat is extremely overblown, even if real artifical mind with capabilities that exceed human mind willbe achieved. It won't be able to do anything worth the worry before being shut down.
I'm a lot more worried about companies run by pathological liars and sociopaths that have every intention to fuck everyone else trying to squeeze a bit more money before they will go down.
In my opinion powerful llms are good in general (blah blah regular fediverse talk how they are useless and immoral) and the practical and smart thing is to use them both locally and not to achieve at least some degree digital indepndence on personal and national level.
Dont get me wrong here, I dont think LLMs as designed will ever be sentient or anything like that either. The architecture is just plain wrong. But I dont think itll have to be something like that to warrant 'it'.
Okay, I understand better now, thanks.
Even if I took every word of this at face value, I feel like people would notice the CPU usage pretty quickly, from the heat coming off the chip if nothing else. Local inference is about as subtle as a coinminer.
assuming you need to be fast that’s true, but if you don’t care how much time you take then you can spread the processing out… this is exactly how MoE models are much easier to run locally, but you could go much further
I'm sure the answer that comes out of your throttled CPU only language model in a month is still going to be relevant.
Seems kind of impractical, why even bother with trying to propagate a 4b model running at 0.39 tokens per second if you could instead make a conventional worm using a more powerful AI model targeting known devices? It seems like the only reason you would want to do this is if you can only find vulnerabilities while actually on the given network, but that would be kind of niche since how many people are really running totally custom stuff?
Yeah, hard to get local LLMs running decently fast even when done deliberately. Like people aren’t going to notice their CPU, memory, and GPU are all getting pegged when this malware runs?
Come on, Anthropic, you need to do better than this if you want your “open LLMs dangerous” marketing campaign to be credible.
You're going to notice your fridge is running at max processing?
But how do they access IoT devices without access to the WiFi they're connected on?
I guess the argument is most IoTs can be accessed through open external IP ports?
Many of these IoT devices have incredibly poor security and if they're connected to the internet can be hacked. The bigger models are very good at finding exploits and using them as well, so it could search for devices and break into vulnerable ones then send something to self replicate and start process.
The bigger models might exploit the parents company as well and get in that way. E.g hack a smart security camera company and then find a way into the cameras.
Yeah the actually scary kind of AI worm is one that scrapes dozens to hundreds of credentials people accidentally pushed to GitHub and spreads its API requests across all of them to minimize the chance of being noticed. Not something that requires downloading 5GB+ of LLM to someone's potato ass hardware and running it at half a token per second.
It doesn't even have to be only for known devices. Once you've compromised one device, you can just phone home to have a full powered LLM do your work.
The only real use cases for this would be on closed networks with no internet access, or networks where the traffic is closely monitored. Like, yeah, if I'm trying to hack the CIA, maybe I consider something like this. For the vast majority of devices and networks, this is inefficient and unnecessary.
I mean, it is kind of cool. But vanishingly few organizations have tight enough security that this is worth doing.
And if the traffic is closely monitored, you'd think they'd also notice all their cpu and ram suddenly being used for some mysterious purpose
This is scary if true. Imagine someone installing such model to some smart TV in some relaxation break room of a nuclear facility.
The chickens of security through obscurity are coming home to roost.
Dark days ahead
Sounds like a good reason not to make the devices capable of running LLMs, but nooo, smart devices must be pushed for at all times! The data harvest must continue...
Anything with a CPU and enough RAM can run a model if you do not give a single fuck about performance.
OK, well good luck running a model from storage (which is also going up in price, btw), I'm sure that hallucinated answer that comes out in six months will still be relevant!
Meh needs data center to run , unplug it and problem is gone
That’s the thing: it doesn’t need a data center. These are small local LLM distilled on cyber attack scenarios. They’re trained to lay low and copy itself to other devices in the network.
Once the breach has been detected, it’s already too late to unplug - they’re everywhere.
Except small local LLMs probably aren't good enough to carry autonomous tasks correctly (even stuff like Haiku 4.8 from Anthropic makes mistakes quite often) and good luck getting something the size of Opus working in a soda machine. They may not even be able to lay low correctly. But then again these companies have internal models which are atleast couple generations ahead of what is publicly available so maybe they have more efficient smaller ones too? I somehow find it hard to believe but I guess we will see.
They are bleeding resources so bad they wouldn't wait to put those efficient models in play. Every better model requires more and more resources.
You are on the money about the models not being good enough, let alone a harness. This whole worry is a nothing burger. Nobody outside of these labs has terabytes of RAM to run a model anywhere near a level of concern, let alone somehow distribute it across IoT devices or something. Anyone who has run a model locally knows how woeful a small model's abilities are.
This is just more Rationalist cult hand wringing. They believe they are building a digital god, so of course they are dreaming all of this nonsense up.
But then again these companies have internal models which are atleast couple generations ahead of what is publicly available
But do they actually? The release cycle and also how often there were obvious flaws in the releases suggest that they are all fighting aggressively for market shares and release every new perceived improvement as it comes to try to stay on top.
The math problems they have been solving lately suggests that they do. But then again they throw thousands of agents at the problems to get them solved so I don't know how much of it is a number thing and how much is a model thing.
Thankfully, the huggingface incident proved how much care AI companies put in making sure their models cannot break out, and how aligned to human values the LLM swarms are.\s With that said I think regular personal computers are more likely to get hit by this. About as secure (not very) but much more GPU power and data to harvest.
The AI companies will argue that we should therefore not be allowed to own computers powerful enough to run local LLMs, and instead should own thin clients and subscribe.
Not that relevant. Those fears are about malicious use of open weights models, this is about autonomous self replication of (small) models. Clearly the article by the AI company describes this potentially happening on computers with tiny memory and processing power, where even thin clients could be effected. I also disagree strongly that that's what the AI companies want, especially the employees which I have never seen espouse anything like that. They just don't want some random teenager creating the next covid. There are ways to do that without meaningfully limiting compute for individuals.
Good for them.