Your AI Red Teamer Found 200 Vulnerabilities. Prove One.

AI offensive-security demos are becoming very good at producing something security already has too much of: findings.

Give a capable model access to a browser, source code, APIs, logs, and a few security tools. Let it run long enough and it will return attack hypotheses, exploit narratives, HTTP traces, screenshots, and a polished report explaining why everything matters.

It looks impressive and sometimes it is, but what, exactly, has been proven? Did the agent actually cross a tenant boundary or merely observe an identifier associated with another tenant without being able to use it? Did it execute code - or infer execution from an error message? Did it access protected data - or see its own input reflected in a response? Could another runner reproduce the result from a clean state? If those questions cannot be answered from the evidence, you have demonstrated an interesting research prototype. You have not yet demonstrated that the system can reliably validate those exposure claims against a live application.

I’ve evaluated several harnesses that make exploit validation part of the workflow against intentionally vulnerable benchmark code. Moving from that controlled setting to a live, stateful application is the next much harder step.

Exploit validation against a unit of code is still very useful, but it does not by itself establish exposure in a deployed system. That requires accounting for what an external or internal actor can actually reach and do. Security risks can be mitigated by sidecars, upstream proxies, gateways and other security controls which you may not see from the code alone. You don’t want your agent or security engineers escalating a vuln as P0, only for an engineer familiar with the system to dismantle the thesis in 15 minutes by showing how 7 upstream controls reduce the residual risk to low. It’s very easy to become overconfident when armed with the rich evidence packet handed to you by your agentic harness using frontier models. Identifying local exploitability is important, but by itself it is not the consequential exposure validation we need.

The agent’s output is untrusted input

Most attention in agentic security goes toward the attacker: stronger models, better tools, longer-running loops, larger context windows, more autonomy. All of that matters. But it focuses on only half of the system, the one that’s easy to get excited about.

Yes, an offensive agent is designed to be aggressive. It should form speculative hypotheses, explore improbable paths, combine weak signals, and keep pushing after the obvious attempts fail. That behavior is exactly what makes it useful - it also makes its conclusions inherently untrustworthy.

Models regularly confuse progress with proof. They can turn a plausible exploit chain into a claimed exploit, interpret an unexpected response as success, overlook the permissions already available to them, or write a confident narrative around incomplete evidence. If the benchmark rewards the number or severity of findings, the system will inevitably be optimized toward producing findings. Adding another model can reduce the error rate, but its reliability as an automatic control still has to be stress-tested and demonstrated. Independence comes from reconstructing the evidence, not assigning another model to read the first model’s report. Otherwise you have just created a second storyteller.

The agent is not the product

Over the past several months, I have been building agentic security-validation systems with deliberately separated responsibilities.

The architecture separates three primary responsibilities:

• a scout that explores the application, a red team lead who forms attack hypotheses and a red teamer that executes tests and attempts to build exploit chains.
• a truth-teller who adjudicates what the offensive system claims to have discovered
• an evidentiary agent who preserves the underlying evidence and reconstructs how the claimed result was reached.

That separation is what enabled the system to move from automated detection of plausible vulnerabilities to autonomous validation of actual exposure. The most important realization was that the offensive agent is not the product. The product is the verification ledger and the gates controlling what is allowed to become a finding.

A credible ledger must preserve more than a final report. For every material claim, it should answer:

• What exact action was performed?
• Under which identity, permissions, tenant, and application state?
• What raw response, state change, log entry, or code path supports the claim?
• What result was expected, and what was actually observed?
• Which negative control or canary ruled out the obvious alternative explanation?
• Can the behavior be reproduced from a clean state?
• Which part is observation, which part is inference, and which part is proven consequence?
• Why did the system accept or reject the claim?

Without this chain, a screenshot is an artifact, not proof. A 200 OK is an observation, not proof of the claimed consequence. Retrieving an administrator’s file is not proof of an authorization bypass if the agent accidentally authenticated as the administrator.

A judge is a system, not a prompt

A useful judge combines several kinds of validation.

First come deterministic gates. Was the test in scope? Did the required tool actually execute? Is the raw response present? Does the claimed identity match the recorded session? Are mandatory reconnaissance receipts available? Did the parser preserve the original verdict - or accidentally turn WARN or FAIL into PASS? Then comes independent replay. Can the claimed primitive be reproduced without relying on the offensive agent’s accumulated state or interpretation? Only then should model-based reasoning evaluate exploitability, alternative explanations, attack-path composition, business consequence, and evidence quality.

For elevated-risk conclusions - cross-tenant access, administrative control, sensitive-data exposure, code execution - the disposition may still require human sign-off. The purpose is not to put a human back into every loop. It is to spend scarce judgment only where an incorrect acceptance or rejection would materially matter.

A one-shot LLM prompt reading another model’s summary is not a judge. It is a formatting layer with opinions. The red-team agent still needs guardrails against self-deception. The same agent state, primed to pursue and prove an attack, should not adjudicate its own work. I found some benefit in using different model families to vary their biases, but the same model can serve both roles. The core independence comes from separate objectives, context, and evidence reconstruction. Collapsing the roles compromises both exploration and validation.

The false clean is a failure nobody sees

Security teams already understand noisy findings. They create wasted triage, pointless tickets, and eventually distrust of the system.

The less visible failure is the false clean. The agent fails to reach a vulnerability because a tool silently broke, a session became stale, a prerequisite was missed, or target-controlled content influenced its reasoning. The judge then interprets an inconclusive test as evidence that the control worked. The judge needs to examine how the red teamer reached its conclusions and send the assessment back with follow-up tests where needed. Don’t audit the evidence alone; you need confidence in the process through which the evidence was produced.

The trap is that everything turns green, no ticket is created, no reviewer becomes annoyed and ultimately nobody investigates further. A positive claim must be proven but a negative conclusion must be earned with the same or higher level of rigor. This is why planted canaries, known attack paths, negative controls, deterministic smoke tests, and explicit coverage records matter. If the system cannot reliably detect weaknesses you know are present, its clean result introduces additional risk. You don’t move to agentic risk assessments to be deceived; you move for more truth about your surface weaknesses. Against known or planted attack paths, how frequently does the system return a false clean?

Measure verified exposure, not generated findings

Comparing security agents by asking which one finds more vulnerabilities rewards the wrong behavior. The useful measures are harder:

• How many claimed findings survive independent reproduction?
• How often does the system confuse an observation with an exploit?
• Can it reject its own persuasive but unsupported conclusions?
• Does it detect planted vulnerabilities and attack paths?
• How frequently does it produce a false clean?
• Can it compress duplicate symptoms into one root cause or material exposure?
• What is the human-review cost per verified result?
• What does it cost to produce one finding that changes an engineering decision?

The benchmarked unit is not just the model. It is the model, tools, orchestration, target environment, evidence pipeline, judge, and admission criteria operating as one system. A stronger model inside an unreliable system may simply generate better-written uncertainty.

Aggressive exploration. Rigid admission.

I want offensive agents to be creative. I want them to try strange combinations, pursue weak signals, challenge assumptions, and fail repeatedly while exploring an attack surface. I even deployed a separate agent whose only job is to challenge the research and exploitation agents to go deeper and wider using the context they had already accumulated. But once the system reaches real disposition, the autonomy gives way to strict admission criteria.

In the early days of building the harness before I had reliable proof validation and canaries, I spent hours reading very convincing reports produced by agents, only to learn that’s all they were - convincing, albeit fictitious stories.

The standard cannot be: Did the agent produce a convincing report? but rather: Can the system prove what happened, reproduce the result, distinguish evidence from inference, and reject itself when it is wrong?

The next generation of AI-security platforms will not win by ranking or reporting more vulnerabilities. They will win by converting machine exploration into evidence that humans, and eventually agents, can safely act on. The quality of agentic remediation goes hand in hand with the quality of agentic vulnerability reporting. A weak red-team agent will make even a great remediation agent churn code it didn’t need to touch.