Somebody Has to Run the Exploit

Two piles

A 9.8 for code that never existed

Fifty-four of 55 security advisories published by a single GitHub account were fabricated. JFrog's Afek Berger published the audit on July 30, 2026. Their CVSS scores ran from 7.5 to 9.8, one briefly at 10.0.

He cloned sqlite/sqlite, checked out the tags named in the advisories, compiled the releases in Docker containers, and fed each proof-of-concept SQL into the binaries under AddressSanitizer. Nothing crashed. In several cases the code the advisories described did not exist in the versions they named.

An afternoon of work. Nobody in the chain that published those advisories had done it.

The same week, Google's Chrome security team posted that “in the last two milestones, Chrome 149 and 150, we have fixed 1072 security bugs, surpassing the total number of security bugs fixed across the prior 23 milestones combined.” They credit AI throughout: “at this point, we have LLMs generating candidate fixes for most vulnerabilities.” The post never separates AI-found bugs from fuzzer- or human-found ones, and never defines what counts as a security bug. Directional, not a measurement.

Same technology. Same seven days. The difference: somebody ran the exploit.

The evidence

The machines actually got good

Anyone still calling these tools useless stopped reading the evidence sometime last year.

Anthropic's Project Glasswing update on May 22, 2026 reported 23,019 findings across open-source projects, 6,202 rated high or critical. Of those, 1,752 were “carefully assessed by one of six independent security research firms,” and 90.6% (1,587 of them) “proved to be valid true positives.”

Epoch AI's Luke Emberson counted 371 critical and 1,175 high-severity CVEs from 21 major vendors in June 2026 alone, “more than 3.5× the monthly record prior to Mythos' release.”

HackerOne's April 21, 2026 platform data shows submissions up 76% year over year to a record March, and “about 25% of findings were confirmed exploitable, a rate that has held steady despite the surge in submissions.” If this were a slop wave, that rate would have collapsed. The flood scaled the signal instead.

In April 2026 the Rust project merged fixes for an out-of-bounds write in CString::clone_into() and a heap overflow in slice::join, both surfaced by Mythos. Rust. The language whose entire premise is memory safety.

Daniel Stenberg, curl's lead developer, in May 2026: “AI powered code analyzers are significantly better at finding security flaws and mistakes in source code than any traditional code analyzers did in the past. All modern AI models are good at this now.” And: “Not using AI code analyzers in your project means that you leave adversaries and attackers time and opportunity to find and exploit the flaws you don't find.”

The funnel

Five findings became one

Same post. Stenberg was offered model access through the Linux Foundation: “I signed the contract for getting access, but then nothing happened.” Access kept slipping, so someone who already had it scanned curl instead. The report claimed five “Confirmed security vulnerabilities.”

I think using the term confirmed is a little amusing when the AI says it confidently by itself. Yes, the AI thinks they are confirmed, but the curl security team has a slightly different take.

After hours on the list of five, his security team was left with one. Three were false positives flagging documented API behaviour. The fourth they deemed “just a bug.” The survivor became a severity-low CVE in curl 8.21.0.

Four of five confident claims were wrong. Small sample, and against one of the most audited C codebases on earth, so read it as an anecdote rather than a rate. Whatever verification happened upstream did not remove the noise. It only decided who sorted it: Stenberg and his volunteers, for hours.

In fairness, the same report contained about twenty non-vulnerability bugs, “described and explained very nicely,” with “barely any false positives.” curl got better. Somebody just paid for it in unpaid evenings.

The measurement

The harness matters more than the model

In June 2026 a team at Semgrep ran the experiment: Katie Paxton-Fear, Seth Jaksik, Brenden Noblitt and Erik Buchanan held everything constant except the scaffolding, same vulnerability class, same dataset.

Their harness scored 61% F1 running GPT 5.5 and 53% running Opus 4.8. Every bare-prompt run landed below both: GLM 5.2 at 39%, Claude Code at 37% and 28% across two variants. GLM's two-point edge over Claude Code is well inside the noise, and the authors do not lean on it.

Their conclusion: “the harness still matters more than the model.” And the diagnosis: “the largest performance gap in the table isn't between models, it's between configurations that get endpoint discovery and those that don't.” Everyone spent the year upgrading the model. The upgrade that mattered was the plumbing.

A commenter on the Chrome thread compressed it on July 31, 2026: “Your LLMs are as good as the loop they run in, and the loop is as good as the verifier.”

Architecture

What it looks like when someone pays

Cloudflare's Grant Bourzikas described their pipeline on May 18, 2026: “Ask a model to find bugs, and it will find them, whether the code has any or not.” Their answer: “An independent agent re-reads the code and tries to disprove the original finding. It uses a different prompt and has no ability to emit new findings of its own.” Why: “putting two agents in deliberate disagreement is way more effective than just telling one agent to be careful.” The acceptance criterion: “a finding that arrives with a PoC is a finding you can act on.”

Elastic published its own numbers on August 4, 2026, one day before this piece — self-reported and unaudited, but with the methodology attached. Ioannis Kakavas describes an eight-stage analysis: a self-challenge review, an adversarial pass that scores CVSS blind before reading the original analysis, then reproduction inside a throwaway Docker environment.

What that buys: “a typical report costs roughly $2 to triage,” agreement with human security engineers 85% of the time “validated against 764 known-outcome reports,” and precision that climbed from 52% to 68% over five calibration cycles.

Two dollars. That is what it costs Elastic to find out whether a stranger's report is real, before a human spends an hour on it.

Mozilla's engineers tried static analysis with an LLM, then abandoned it because the “high rate of false positives made them impractical to scale.” The replacement: a dynamic harness that “can find real bugs and dismiss unreproducible speculation,” with severity assigned by the AddressSanitizer crash class, not the model's opinion of itself.

• Argue with itCloudflare runs a second agent on a different prompt whose only job is to disprove the finding, and which cannot raise findings of its own
• Run itElastic reproduces surviving reports inside a throwaway Docker environment before a human is involved, at roughly two dollars a report
• Let the crash decideMozilla assigns severity from the AddressSanitizer crash class rather than from the model's opinion of its own finding

Google's triage checks for a proof of concept and attaches stack traces before a human is involved. The result: “we estimate that this new process is saving hundreds of hours of developer time per month,” a claim they hedge themselves with “while it's hard to measure precisely.”

The chain

Four organizations and nobody ran the code

Red Hat assigned one of the alleged use-after-frees a perfect 10.0. The records entered the National Vulnerability Database with CISA-supplied enrichment. MITRE eventually rejected the whole repository. Brandon Vigliarolo reported in The Register on August 3, 2026: four organizations touched a fiction, and none compiled the code.

Alan Coopersmith, an Oracle Solaris engineer, explained why on the OSS-Security mailing list: “MITRE and most other CNAs which assign CVEs for code they don't produce themselves operate on the honor system, and trust CVE requesters to have verified the information they provide. The CNA is often not in a position of being able to verify the report themselves.”

Glauber Costa framed the economics when Turso retired its bug bounty on May 12, 2026, having paid a grand total of five people:

The main problem of course is that it costs the slopmaker perhaps a minute to generate their submission. But it costs us hours to read, understand, and engage with them.

A reproducer is an invoice. Skip it and you mail the bill to whoever opens the report. Stenberg said the same thing in June 2026, less gently: without a reproducer “you instead push that work to the receiving end. We still need the reproducer. We still need a test case.”

One Hacker News commenter posted the exact wording of a cyber-insurance policy voiding coverage for any breach exploiting a critical CVE where “a patch or fix... has been available for 21 days prior to the date of the incident and has not been applied.” A fabricated 9.8 has no patch to apply, and the clock runs anyway. Another commenter described being handed a scanner's output listing Bluetooth stack vulnerabilities for datacenter servers that have no Bluetooth. “But they just didn't care.”

On May 1, 2026, Google cut Chrome's base memory-safety reward to $500 (some payouts dropped roughly tenfold) while raising a zero-click Titan M exploit to $1.5 million. Their stated criterion: “we now consider the most effective reports to be concise, containing only a reproducer and the necessary artifacts to help us validate and route the issue.”

The bottleneck

Fixing is the part nobody automated

Glasswing again. 90.6% true positives, six independent firms. And then: “75 of the 530 high- or critical-severity bugs we've reported have now been patched.” Fourteen percent. Anthropic adds: “several maintainers have told us they're currently severely capacity constrained, and some have even asked us to slow down our rate of our disclosures.”

HackerOne, from the platform side: submissions up 76%, and “remediation is not keeping pace, improving just 19% year over year.” Verification was the bottleneck. Automating it moved the bottleneck downstream, to the only part of this pipeline still made entirely of people.

Findings that were real

90.6%

of 1,752 high and critical findings assessed by six independent security firms

Reported bugs since patched

75/530

high and critical bugs disclosed by Project Glasswing, as of May 22, 2026

Submissions vs. remediation

76 / 19

percent year-over-year growth on HackerOne, reported April 21, 2026

Greg Kroah-Hartman told The Register on March 26, 2026 that the kernel's slop problem evaporated: “Months ago, we were getting what we called ‘AI slop,’ AI-generated security reports that were obviously wrong or low quality... Something happened a month ago, and the world switched. Now we have real reports.” Asked what changed: “We don't know. Nobody seems to know why.” His “a month ago” points at roughly February.

No new harness explains that. Stenberg's April 2026 numbers point the same way. Report volume was roughly double 2025, itself more than double the years before, and the confirmed-vulnerability rate returned to and exceeded pre-AI 2024 levels, “somewhere in the 15-16% range,” with “almost every security report now” using AI.

This does not retire the Semgrep result. A harness improves the ratio; it does not drive the residue to zero. Capability decides how big the pile is. The harness decides whose desk it lands on. Even at curl's recovered 15–16%, five reports in six are read for free by volunteers, on double the volume.

The bill

What the bliss taught everyone

So curl stopped.

The maintainers took July 2026 off from vulnerability reporting. Stenberg's retrospective, published August 3, gives “the single word answer” up front: fine. One report reached his private inbox in thirty-one days. “I just ignored that email. That was easy.”

They got back their project: “we felt free, unchained, and now suddenly able to do what we wanted.” They worked on parts of the codebase they had abandoned because vulnerability reports “sucked all the air out the room.” CNA rules obliged them to respond to critical reports within 72 hours. Nothing arrived that needed it.

On the Reddit thread about the blackout, the top comment at 674 points: “They're taking a vacation from unpaid support, and if that vacation is scary to anyone, they're more than welcome to establish a paid support contract. We aren't owed free volunteer maintenance from open source supporters.”

Stenberg kept supporting paying customers throughout, and expected that an announced month of silence from the maintainers of software running in twenty billion installations might sell a few support contracts.

This did not happen — at all. By this I think we should conclude that (commercial) curl users were not worried either.

Nobody paid. They never have. The industry has spent decades running on a verification subsidy donated by people it does not employ, and 2026 is the year the invoice got large enough to read. Two dollars a report, or somebody's July.

Run the exploit before you file it, before you score it, before you pass it on. It was always your job.