Vulnerability Research Is Cooked (sockpuppet.org)

There is ablog post on sockpuppet.org arguing that we are not prepared for theupcoming flood of high-quality, LLM-generated vulnerability reports andexploits.

Now consider the poor open source developers who, for the last 18months, have complained about a torrent of slop vulnerabilityreports. I'd had mixed sympathies, but the complaints were at leastempirically correct. That could change real fast. The new modelsfind real stuff. Forget the slop; will projects be able to keep upwith a steady feed of verified, reproducible, reliably-exploitablesev:hi vulnerabilities? That's what's coming down the pipe.

Everything is up in the air. The industry is sold on memory-safesoftware, but the shift is slow going. We've bought time withsandboxing and attack surface restriction. How well will thesecountermeasures hold up? A 4 layer system of sandboxes, kernels,hypervisors, and IPC schemes are, to an agent, an iterated versionof the same problem. Agents will generate full-chain exploits, andthey will do so soon.

Meanwhile, no defense looks flimsier now than closed sourcecode. Reversing was already mostly a speed-bump even forentry-level teams, who lift binaries into IR or decompile them allthe way back to source. Agents can do this too, but they can alsoreason directly from assembly. If you want a problem better suitedto LLMs than bug hunting, program translation is a good place tostart.