this post was submitted on 26 May 2026
65 points (93.3% liked)
Fuck AI
7069 readers
1970 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
AI isn't particularly good at finding bugs and vulns. It's just that barely anyone except the devs have ever looked at the source of most open source, and for the first time there's automated mass code review. And a lot of open source projects are kinda shit, unmaintained experiments that no one ever reviewed.
I don't mean to say it isn't finding bugs, it's just that the quality of the reports are often low, there's way more noise than signal, and there's always been low hanging fruit with random open source projects. I mean, half the time a big new vuln was announced is because some researcher finally sat down and took the time to look at something. Massive software projects were hard to sift through in an automated and repeatable fashion.
You might want to look at the story about the Mythos model, apparently it is particularly good at finding vulnerabilities.
It's really not. Out of the 23 000 something vulnerabilities claude claims to have discovered in open-source projects only 3-500 have been reported to repo owners,, only 65 have been confirmed and given any rating at all. This is not any more efficient than any other form of fuzzing, they just did a whole lot of it.
I’m struggling grasping your logic. I am very far from being an AI fanboy but I’m also not a luddite.
So we have tools now that can pretty much autonomously scan through any accessible codebase and find new vulnerabilities that were not found before. And you say that’s not a big deal because anyone could have found those vulnerabilities if they looked?
Of course, that’s the whole point, nobody was able to attack at that scale before, and now many actors are. Your argument reminds me of what was common to hear 15 years ago when nobody secured anything: “why would I complicate my life with security, nobody wants to hack me! and if one day the CIA decides to come after me, they can get through security anyways!” True, until you have botnets scanning every ip…
The problem is that not "many actors" are able to attack at this scale, because running a scan at this scale is extremely expensive. If I were to run a thousand fuzzers on a piece of code I will almost certainly find a vulnerability, but I can't do that because of the prohibitive cost. Anthropic is essentially buying marketing by doing this to make their product seem more useful than it is.
Got it, that makes sense.
The problem isn't that it's finding stuff. It's that it's also finding a ton of useless crap that a human has to sort through because the machines aren't reliable. If you get blasted with 100 new lengthy and overly detailed bug reports vomited up by a text generator and you have to triage them all to figure out if there even is a needle in that haystack, the added benefit is practically nullified by the overhead of actually utilising it.
Oh I know the response to this: you have to set up an agent team to triage the reports!
I understand why a team wouldn’t want to have anything to do with AI. I don’t understand why a user thinks software is compromised if they accept AI generated bug reports.
For some, it may be a matter of trust: If I don't trust AI code, but you do, I don't trust you either.
For others, it will be a matter of hardline principles: If I don't want AI to get any foothold whatsoever, but you accept it in some form, you join the trend I oppose and I don't want to associate with you or contribute to the popularity metrics of your product (such as unique downloads).
I don't feel like discussing the merits of either stance, but I hope this helps you understand the premises leading to that conclusion at least.
Those are just the ones that Mythos has claimed so far. They stated that is only about 1% of all the vulnerabilities they discovered and were publicly announced. Firefox 150 had over 270 bug fixes, with 13 of them as high severity.
Mythos is also finding high severity vulnerabilities that have been in systems for over 20 years with no humans able to discover them during that time. Its patient, and can look at the entire repo and how it all works together.
The problem is that I do not believe a word that anthropic says. They say this is only 1%, but do they have any proof to back it up? I am also sceptical of the claim that it can "look at the entire repo and how it all works together". It can produce an approximation which could give it an advantage over more traditional fuzzers, but most reported bugs are still very local(and/or non-existant) and easily ruled out if it could actually model the naur theory behind the code.
They already explained how they have placed hashes inside all their bug reports for Project Glasswing and will reveal their report once there has been time for patches to be applied.
Mozilla, developer of one of the most active and heavily scrutinized open source repositories in existence today, blogged about it with their product known as Firefox. They agree with you that it doesn't do anything better that what a human researcher could find, but its perk is that it can relentlessly play that role and keep looking, while human researchers have to sleep, eat, and enjoy other activities:
https://blog.mozilla.org/en/privacy-security/ai-security-zero-day-vulnerabilities/
I'll see when these hashes materialise, until then I have to assume LLM companies are lying always about everything.
See, the problem is that I am not talking about human researchers, I am talking about other methods of automated fuzzing. I believe mozilla is overstating how useful the LLM has actually been. This has many reasons, one of them being that their main source of income is trying to become an LLM company. If that project fails said company might have to make some unfortunate cuts.
Im sure FreeBSD probably appreciates the bug reports as well, and I don't believe they are tied to LLMs. They have totally revamped their processes recently to accommodate for the influx of reports coming in.
Nor should you.