How we grade evidence
Most AI security coverage blurs the line between something demonstrated once in a lab and something attackers are actually doing. Every claim here carries a grade, and the grade is assigned by a named person.
The levels
- CONFIRMED IN THE WILD
- Trustworthy evidence of real-world exploitation.A specific, named source describing observed exploitation — not a vendor’s characterisation of risk.
- REPRODUCED
- A demonstration verified independently.We name who reproduced it. “Researchers reproduced” is not sufficient.
- PUBLIC POC
- A public proof of concept exists.A link to the PoC. Not independently reproduced by us.
- VENDOR REPORTED
- A vendor asserts it.The vendor’s own advisory, with no independent verification. The most commonly overstated level.
- RESEARCH HYPOTHESIS
- Theoretical or experimental.A paper or writeup. Not established as real-world exploitation.
- DISPUTED
- Credible conflicting evidence.Both sides cited.
- UNVERIFIED
- Insufficient evidence.Nothing. This is the honest default, and we use it freely.
Rules that follow
A grade requires a named human. No model assigns an evidence level here. That isn’t a stylistic preference — the database refuses to store a grade without a named grader and a timestamp, and the site refuses to build a report without at least one source.
Evidence level is not priority. An important story can beUNVERIFIED; a minor one can beCONFIRMED_IN_THE_WILD. We keep them separate so that prominence never implies verification.
We default to the weaker level. If a claim could beREPRODUCED or PUBLIC_POC, it isPUBLIC_POC until someone is named.
AI involvement is never inferred. A vulnerability appearing in a conventional security feed says nothing about whether AI was involved. Treating those as the same thing is how publications end up announcing an “AI attack wave”.
Nothing publishes automatically. Candidate stories are found and structured by software running locally. Judgment is not delegated, and an unreviewed draft is never published with a disclaimer attached.
When we get it wrong
Grades change as evidence arrives. A correction carries a date and the prior version is preserved rather than quietly overwritten. Seecorrections.