Engineering

Vendor-neutral guidance for CTOs, CISOs, and engineering leaders on AI-generated code security, measurement, governance, DORA metrics, technical debt, and engineering health. Not legal advice. Does not certify your codebase or determine regulatory applicability.

Engineering metrics, explained for leaders

Measuring software delivery well is less about dashboards and more about measuring the right things in a way teams can't game. The most durable reference is DORA (DevOps Research and Assessment), whose Accelerate research program — summarized annually in Google Cloud's State of DevOps report — established that a small set of delivery metrics correlate with both throughput and stability, and that the two are not a trade-off: elite teams ship faster and break less. The value of a metric is in the behaviour it drives, so every number below pairs with a failure mode you should watch for.

The four DORA keys

DORA tracks four measures, split into throughput and stability:

  • Deployment frequency — how often you ship to production. Throughput. Elite teams deploy on demand (multiple times a day); low performers deploy between once a month and once every six months.
  • Lead time for changes — the time from code committed to code running in production. Throughput. Elite is under an hour; low is more than six months.
  • Change failure rate — the share of deployments that cause a degraded service needing remediation (a hotfix, rollback, or patch). Stability. Elite teams sit in the 0–15% band.
  • Failed deployment recovery time (formerly "time to restore service") — how long it takes to recover from a failed deployment. Stability. Elite is under an hour.

The 2023 and 2024 State of DevOps reports kept these four as the core while adding a reliability dimension and a stronger emphasis on the role of platform engineering, developer experience, and documentation quality as upstream drivers. The performance bands (elite / high / medium / low) shift year to year, so treat any specific threshold as a snapshot of the cohort that responded, not a fixed law.

Why two of the four are stability, not speed

The frequent misread of DORA is to chase deployment frequency and lead time alone. Those are throughput; on their own they reward shipping faster even when quality slips. Change failure rate and recovery time are the counterweight — they catch the team that "ships fast" by pushing breakage to production. A healthy reading moves all four together: you are deploying more often and your failure rate is flat or falling. A rising deployment frequency with a rising change failure rate is not progress; it is debt accruing in production.

DORA is not a productivity ranking

DORA measures the delivery system, not individuals. Using it to rank or compare engineers is the fastest way to destroy the signal: people optimize the measured number (splitting PRs to inflate deployment counts, closing incidents prematurely to lower recovery time) and the metric stops reflecting reality — a textbook case of Goodhart's law. This is why DORA's own guidance, and complementary frameworks like SPACE (Satisfaction, Performance, Activity, Communication, Efficiency), insist on measuring at the team or system level and pairing hard delivery metrics with developer-experience signals. The honest use of DORA is a trend line for a team, read alongside context, not a leaderboard.

Measuring AI-assisted development responsibly

As coding agents write a growing share of code, the temptation is to measure "AI productivity" by lines or acceptance rate. Those are activity counters, not outcomes — more accepted code can mean more review burden and more latent risk, not more value delivered. The responsible frame keeps DORA and quality metrics as the outcome layer (is the team shipping stably and safely?) and treats AI-specific signals — review coverage of agent-generated changes, defect and revert rates on that code, time spent reviewing versus writing — as diagnostics underneath. The question is never "how much did the AI write" but "did delivery get faster without change failure rate or technical debt rising."

Metrics only help if they resist gaming

Every metric here can be gamed if it becomes a target in isolation. The defences are structural: measure systems not people, read trends not single points, pair throughput with stability so neither can be optimized at the other's expense, and keep the numbers derived from systems of record (version control, CI/CD, incident tooling) rather than self-reported. A number a team can quietly inflate is worse than no number, because it manufactures false confidence.

This hub is operational guidance distilled from named public sources — the DORA / Accelerate research program and Google Cloud's annual State of DevOps reports, and the SPACE framework for developer productivity. Performance bands reflect the cohort of each year's report and shift over time. This is engineering guidance, not legal advice, and it does not certify your codebase or determine regulatory applicability.

Every engineering guide in this cluster

Is AI-generated code secure?

Evidence on AI-generated code security: failure modes, empirical study caveats, OWASP and NIST SSDF guidance. No absolute secure or insecure claim. Not legal advice.

How do you measure AI-generated code risk?

A defensible framework to measure AI-generated code risk: signals, metrics, and trend thresholds (recommendations, not standards). OWASP, NIST SSDF, DORA. Not legal advice.

How can you detect AI-generated code?

How AI-generated code can and cannot be detected: provenance and telemetry versus probabilistic classifiers, with false-positive and false-negative caveats. Not legal advice.

How do you measure human vs AI-generated code?

Measure human vs AI-authored code share with an explicit denominator (lines, commits, or PRs). Pitfalls, Goodhart warnings, and an illustrative example. Not legal advice.

Does AI coding increase technical debt?

DORA, GitClear churn, SEI debt literature, and Accelerate on AI-assisted coding and maintenance cost — with correlation vs causation caveats. Not legal advice.

How do you measure AI coding ROI?

AI coding ROI: licences, review, remediation vs throughput and cycle time — vanity-metric traps and an illustrative example. SRM figures are estimates. Not legal advice.

How do you measure AI developer productivity responsibly?

AI-era developer productivity with DORA and SPACE at team and system level — not individual ranking. DevEx dated; McKinsey/DX as opinion. Not legal advice.

How do you detect security issues introduced by AI coding agents?

AI agent failure modes — secrets, injection, bad deps, insecure config — mapped to SDLC controls. OWASP, NIST SSDF 800-218A, CWE, CISA, KEV, EPSS, CVSS. Not legal advice.

What is an AI coding governance checklist?

AI coding governance checklist: approved tools, provenance, review gates, secrets and IP. Mapped to ISO/IEC 42001, NIST SSDF and AI RMF. Not legal advice.

What should an AI coding policy include?

Editable AI coding policy template: scope, approved tools, data and IP rules, review, provenance, exceptions, cadence. Mandatory vs recommended. Not legal advice.

How should you review agent-generated pull requests?

Review agent PRs: risk-tier table, reviewer checklist, branch protection, SAST and provenance. OWASP, NIST SSDF, DORA, SLSA. Org practice vs SRM. Not legal advice.

How should you measure technical debt?

Measure technical debt with proxy signals — churn, complexity, defects, remediation cost — each with blind spots. No single authoritative number. SEI, ISO 25010, DORA.

How do you measure engineering excellence without gaming the score?

Measure engineering excellence with DORA, SPACE, reliability, and ISO/IEC 25010 — balanced across delivery, quality, and people. Anti-Goodhart guidance. Not legal advice.

What are DORA metrics and how do you calculate them?

The four DORA metrics: deployment frequency, lead time, change failure rate, time to restore. Definitions and worked examples. Not the EU DORA regulation.

How is an engineering risk score calculated and what can you trust?

How ShipReady Metrics builds an engineering risk score: inputs, weighting, denominators, coverage-as-confidence, and how to read it honestly. SRM methodology — not a standard.

How does ShipReady Metrics measure AI ROI and engineering health?

How SRM measures AI ROI and engineering health: Copilot Metrics ingest, AI-authored floor, DORA metrics, committer metering, engineering risk score. Estimates, not audits.

See DORA and engineering health in ShipReady

ShipReady derives DORA keys, change failure rate, and engineering-health trends directly from your source control and CI/CD — system-of-record signals, read at the team level as trends, so the numbers resist gaming instead of inviting it.