Operational guidance, not legal advice. This page distills named public sources (regulator guidance and industry practice). It is not a legal determination, not a notification decision, and not a substitute for your counsel, insurer, or a retained DFIR firm. Verify applicability and current deadlines for your facts and jurisdiction.
How do you measure AI-generated code risk?
Updated
Combine vulnerability findings, dependency exposure, review coverage, authorship share, and change-failure signals into a trended risk view for YOUR repos. Thresholds below are recommendations, not industry standards. Not legal advice.
AI-generated code risk measurement, last verified 10 September 2026 against NIST SP 800-218 and SP 800-218A, OWASP Top 10 and LLM Top 10, DORA and Accelerate research, and CWE/CVSS/EPSS/KEV definitions from their issuing bodies. It is not legal advice and does not set mandatory thresholds for YOU.
Why measure instead of debate
Audience: a CTO, CISO, or engineering leader who needs a repeatable view of AI-assisted development risk. This page is not legal advice. It does not set mandatory thresholds for YOUR organisation and does not replace YOUR risk register.
Anecdotes about AI coding swing between hype and panic. A defensible programme tracks signals over time on YOUR repositories: Do AI-attributed changes introduce more findings? Is review keeping pace? Are dependency and secret issues rising? The goal is trend visibility and prioritisation — not a single score that proves safety.
DORA research (dora.dev, State of DevOps reports) treats delivery and stability metrics as system-level indicators — change failure rate, deployment frequency, lead time, recovery time. Apply the same discipline: measure the system, avoid ranking individuals. Last verified 10 September 2026. Not legal advice.
- Legal requirement versus guidance versus best practice versus SRM recommendation: NIST SSDF practices are guidance. OWASP lists are community consensus. CVSS, EPSS, and KEV are scoring and catalog schemes from FIRST, NIST-supported CVSS, and CISA — use them for prioritisation, not as pass-fail law. DORA metrics are research-backed delivery indicators. ShipReady engineering risk score and AI ROI figures are vendor methodology — recommendations with coverage honesty, not audited standards.
- The is-ai-generated-code-secure guide on this site is the evidence pillar. The measure-human-vs-ai-generated-code guide on this site defines authorship denominators. The detect-security-issues-ai-coding-agents guide on this site maps controls to failure modes. The engineering-risk-score-explained guide on this site describes SRM's composite.
Signals to metrics — mapping table
Map observable signals to metrics YOU can trend monthly or per release. Example thresholds in the table are SRM recommendations for discussion — not NIST, OWASP, or ISO standards. Tune them to YOUR risk appetite and maturity. Last verified 10 September 2026.
| Signal | Metric | How to measure | Example threshold (recommendation only) |
|---|---|---|---|
| Security findings on AI-attributed changes | Findings per 1,000 lines or per AI-attributed PR | Tag findings by commit/PR authorship marker; dedupe across SAST, DAST, secret, and dependency sources | Investigate if AI-attributed PR rate exceeds human-only baseline by more than YOUR agreed margin for two consecutive sprints |
| Critical vulnerability exposure | Open critical/high findings with KEV or high EPSS | Prioritise with KEV catalog, EPSS scores, CVSS base scores | Zero open KEV-listed issues on default branch — common organisational target, not a universal law |
| Dependency risk | New vulnerable dependencies introduced per month; mean time to remediate | SCA on every merge; track transitive depth for npm ecosystems | Remediate KEV-listed packages within YOUR SLA (many teams target 7–14 days — YOUR policy) |
| Secret exposure | Secret-scanning hits on AI-touched commits | Pre-commit and CI secret scanners; block merge on hit | Zero tolerated secrets on main — recommendation aligned to CISA secure-by-design |
| Review coverage | Percent of AI-attributed PRs with human approval before merge | Branch protection plus authorship tags | Human approval required on high-risk paths; define high-risk in YOUR governance checklist |
| Authorship share | AI-attributed lines, commits, or PRs over total (denominator explicit) | Git markers, tool telemetry, or IDE attribution — see measure-human-vs-ai-generated-code guide | Use for exposure sizing, not as a performance KPI |
| Change failure rate | Failed deployments or rollbacks involving AI-attributed changes | DORA-style deployment and incident linkage | Compare AI-touched versus baseline cohort; no universal percentage target |
| Remediation cost | Time to fix findings introduced on AI-attributed PRs | Issue tracker plus finding source linkage | Rising MTTR on AI cohort triggers process review — threshold is YOUR call |
How to build the view — steps
Start with provenance you trust: commit markers, IDE attribution, or CI labels — not probabilistic classifiers alone. See the detect-ai-generated-code guide on this site.
Ingest findings from every gate: SAST, DAST, secret scanning, dependency analysis. Dedupe before counting.
Segment at least two cohorts: AI-attributed changes and non-AI changes on the same repos. Compare rates, not raw counts, when volume differs.
Trend monthly. One bad PR is noise; a rising finding rate on AI cohorts is a signal.
Pair with the engineering risk score explained guide for a composite view — with coverage-as-confidence honesty.
Thresholds are recommendations, not standards
No NIST, OWASP, or DORA publication sets a universal numeric threshold for AI-generated code risk. CVSS severity bands are scoring guidance, not merge gates. EPSS is a probability estimate for exploitation, not a compliance test. KEV is CISA's catalog of known exploited vulnerabilities — prioritise them, but YOUR SLA is a policy choice.
ShipReady Metrics may surface suggested bands in product copy or defaults. Treat them as starting points for YOUR risk committee, not audit criteria. Document YOUR chosen thresholds and review them quarterly.
What you need to do now
Operational steps. Not legal advice.
- Pick a denominator for authorship (lines, commits, or PRs) and document it — the measure-human-vs-ai-generated-code guide on this site.
- Connect SAST, DAST, secret, and dependency sources; enable deduplication.
- Baseline human-only finding rates before tightening AI policy.
- Add branch protection requiring human review on AI-attributed PRs to high-risk paths.
- Review AI versus human cohort metrics in a monthly engineering risk forum.
- Record YOUR threshold choices in the ai-coding-governance-checklist guide on this site.
Checklist
Measurement readiness. Not a certification.
- Authorship tagging is consistent across teams?
- Finding sources cover SAST, secrets, and dependencies at minimum?
- KEV and EPSS used for prioritisation, not vanity counts?
- Thresholds documented as YOUR policy, not mislabeled as ISO or NIST mandates?
- Change failure rate tracked for AI-touched deploys?
- Individual developers are not ranked on AI usage?
Where this shows up in ShipReady Metrics
AI-authored-code floor in signed-in AI Code Readiness reporting counts attributed AI commits — exposure sizing, not risk scoring alone.
AI ROI scoring (including Copilot Metrics ingest where configured) pairs cost and throughput estimates — figures are estimates, not audited financials; per-team views use cohort minimums.
Security → Findings ingests multi-source vulnerabilities with KEV, EPSS, and CVSS prioritisation per the what-we-measure methodology.
Engineering risk score composites delivery, quality, and security signals with vendor methodology — not an industry standard or audit opinion. Coverage gaps lower confidence; the product states that honestly.
This product does not set mandatory thresholds for YOUR organisation and does not certify low risk.
Primary sources (last verified 10 September 2026)
NIST SP 800-218 and SP 800-218A (SSDF and GenAI profile). OWASP Top 10 and LLM Top 10. DORA research at dora.dev and Accelerate (Forsgren, Humble, Kim). CVSS specification. EPSS from FIRST. CISA Known Exploited Vulnerabilities catalog. CWE from MITRE. Not legal advice.
Frequently asked questions
What is the official threshold for acceptable AI code risk?
There is no universal standard. NIST SSDF and OWASP describe practices, not numeric AI-risk limits. Set thresholds from YOUR baseline, regulatory duties, and risk appetite. Document them as YOUR policy.
Should we use one metric?
No. A single number invites gaming. Combine findings, dependency exposure, review coverage, and change outcomes. The signals table above is a starting map.
Does the engineering risk score replace this framework?
No. The engineering risk score is an SRM composite for trend visibility. YOU still own threshold policy and control selection. See the engineering-risk-score-explained guide on this site.
Can we compare AI tools using only acceptance rate?
Acceptance rate is a vanity signal — high acceptance can mean more vulnerable code merged. Prefer finding rates and remediation cost on accepted changes.
Published by ShipReady Metrics, an evidence-based technology and compliance intelligence platform. This guide is educational and vendor-neutral.