From Evidence Production to Assertion Validation
A control can be documented, evidenced and tested, and still not be true as stated. The useful AI problem in compliance is testing whether the full claim holds.
One thing I have been trying to get at in my last few posts is that security becomes much more useful once it stops treating a finding, severity score or collected artifact as the end of the job. In AppSec, the real question is whether we can prove material exposure and move from evidence to mitigation quickly enough for it to matter.
Compliance has the same failure mode, just with different artifacts. A control can be documented, evidenced and tested, and still not be true in the way the company believes it is true.
Take a statement such as: “Admin access requires phishing-resistant MFA.”
The policy requires it, IdP config shows MFA enabled, and Jira tickets document the access review performed by the system owners. All of that may be accurate, and an auditor could reasonably conclude that the control is operating. Audit passed, everyone is relieved.
The harder question is whether those artifacts prove the whole claim. Does the requirement also cover local accounts, break-glass access, service accounts, back-office tooling, legacy applications and every other path capable of performing an administrative action? Often, nobody has evaluated the control in those terms. The bigger the scale, the truer it gets. The evidence is genuine, the testing was performed correctly, and nobody intended to misrepresent anything. Each artifact simply proves a narrower statement than the organization thinks it proves.
That is why one of the more valuable uses of AI in compliance has little to do with generating policies, mapping controls to frameworks or collecting screenshots faster. Those capabilities are already commoditized and rarely differentiate vendors. The more interesting problem is taking a commitment expressed in natural language, breaking it into the technical assertions that must be true, and checking whether the architecture, configuration, code, procedures and evidence support the full claim.
The useful assessment output is not: “MFA evidence collected.”
It is: “Phishing-resistant MFA is enforced for workforce administrators using the primary identity provider, but four in-scope administrative paths remain outside that enforcement boundary.”
That statement is more useful because it tells the company what is true, where it's true and what risks remain. The risk can then be triaged, assigned, funded, accepted or corrected.
That is how compliance moves from evidence production to assertion validation. The progression is similar to what is happening in AppSec.
We are moving from “we found a high-severity issue” to “we proved the exposure.”
Compliance should move from “we collected the evidence” to “we proved the commitment.”
In both cases, the shift is from producing artifacts to testing whether the assertion behind them holds. That is the practical answer to the old “compliance does not equal security” debate: compliance creates security value when it tests whether the company’s claims are true.