is it just me, or does the a. i. companies’ recent focus on automating exploit finding read as an “engage with us Or Else” ploy against the projects that wouldn’t take generated code contributions but can’t ignore security issues
Apr 8, 2026, 1:47 AM@joe it's more "this is actually a thing it can do" i feel as fuzzing does produce results
but, well, after the burst of low hanging fruit, i don't expect a regular crop of bugs
Apr 8, 2026, 1:27 PM@pozorvlak @joe alas, "secure programing in C" turns out to be more than just yelling at linux developers
Apr 8, 2026, 1:30 PMto be clear, if you believe openbsd has a lower defect rather than any other of the bsds, you're absolutely being taken for a ride
Apr 8, 2026, 7:23 PM@tef @joe Yeah as neural networks are generally somewhat effective at recognition and classification in a way they aren't with generation, I'm willing to potentially believe this could even find errors in pure code inspection. Problem: linters of such type tend to have a massive false positive rate. Which might be why they're not productizing this, because finding useful exploits winds up wading through *tons* of nonsense that external customers wouldn't put up with
Apr 8, 2026, 7:33 PM@whitequark @tef @joe Additional possibilities:
- They simply hired some really good security researchers to do a Project Zero kinda thing and the secret AI model is essentially incidental.
- They are about to file a large number of low-quality security bugs on items which turn out to be not practically exploitable, things which could have been found with existing tools, or both
Hard to say right now
Apr 8, 2026, 7:39 PM@mcc @whitequark @tef @joe from what I've seen on that front the participants largely fall into three categories: top-1% bug hunters who are there for the payouts, a few security companies who are keen to engage either for marketing or because leadership is personally invested in AI, and AI firms who want to show off their products.
the somewhat unsurprising result is that the pro bug hunters find very shiny bugs, the security companies find some good bugs, and the AI firms report lots of FPs.
Apr 8, 2026, 7:42 PM@mcc @whitequark @tef @joe AI stuff is involved in the process for all parties, and I can't really say how much it aids the bug hunting in those specific contexts, but I think it does make it quite clear that if there is an uplift in bug hunting capability it is a minor factor compared to the security experience and skill of the person driving it.
Apr 8, 2026, 9:33 PM@mcc @whitequark @tef @joe it definitely finds security bugs. There is a large wave of good findings hitting everyone right now (along with a bunch of trash reports, but this can be improved by tightening reporting processes). However, the things that make the news the most, like the FreeBSD kernel exploit chain, are not actually that impressive. maybe impressive for what a computer can do semi-autonomously, but not impressive or representative of exploitation on platforms that have invested in security. They had a 20-gadget ROP chain to allocate a RWX mapping in a kernel with non-functional stack cookies and no ASLR, exploiting a security bug that sat there for 17 years. the only part of this that would make the news if a human wrote it was that the bug sat there for 17 years
@joe it absolutely comes across as a protection racket
@joe the "we found a local privesc in Linux" seemed particularly silly to tout... we have local privesc in Linux at home
Apr 8, 2026, 3:40 AM@joe it’s finding real issues. Anything that finds real issues and costs money will feel like an “engage with us Or Else” situation
I would be amazed if somehow this gets results as good as the best tools + fuzzers combined.
What it does do is level up script kiddies (do we call them that anymore?). A lot of the steps that required real learning can be automated and the output can be crap b/c who cares if it is maintainable?
Challenges like exploiting multiple layers of bugs to get to something really exploitable and actually turning it into a real exploit require real skill, likely not anymore.
My fear is that amount of security flaws in the current LLM tools offers a really nice playground for a worm to spread very quickly. Especially with the agents socializing w/ each other.
I am sure corporate infosec folks are not thrilled and likely have hands tied, well at least until a real exploit happens.
I will change my answer slightly here.
I was assuming source code scanning to find exploits but if they are instead talking about finding exploits via vulnerability probing via stacking all the existing infosec tooling out there or something similar.
That is 💯 believable they have gotten impressively great results there and I was waiting for that to happen. This was the obvious path that lowers the exploitation barrier but my impression was that they were purposefully avoiding this path b/c the consequences are likely pretty grim.
They could justify it using a MAD model like we did during the Cold War. Once you open that gate though.
Reading more on this:
https://www.wired.com/story/anthropic-mythos-preview-project-glasswing/
and it indeed looks like they turned it into an elite hacking tool. Well, that was a choice.
The next few years will be interesting exploit wise.
@shafik as other folks in my replies noted, it might be that there’s an initial wave of low hanging fruit that these models can find, and their efficacy could taper off over time
Yes but the tails are long here. If we assume organizations step up and catch problems quicker than it should flatten out but not sure that is a good assumption.
The motivation for cracking things is way higher than the motivation for a diverse set of developers to keep on their toes.
We have been writing insecure software for a long time since we knew how to avoid a lot of these issues.
