No denominator, no score: a practical check before sharing delivery metrics

Ask the denominator question first

A dashboard says your change fail rate is zero. Before celebrating, ask one question: zero failures out of how many observed production deployments?

Zero failures out of forty deployments is a measured result, subject to the quality of the failure classification. Zero failures out of zero deployments is not. Neither is zero failures when the deployment source never finished syncing.

Treating those states alike makes a polished dashboard less useful than a blank page. The fix is small: publish the measurement boundary beside the number.

Start with a service, not a league table

DORA's current guide describes five software delivery performance metrics: change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate. It recommends applying them to one application or service at a time. It also warns that comparisons between very different applications can mislead.

That gives an engineering manager a practical starting point. Pick one service and one reporting window. Agree on what counts as a production deployment, how hotfixes are identified, and which source owns each event. Don't begin by ranking teams whose deployment processes differ.

A useful first measurement contract might read:

Service: checkout-api. Window: September 1-30. Production environment: prod. A deployment counts when a distinct deployment event reaches a successful terminal state. Retried records with the same deployment ID count once. Change failure means that deployment required immediate intervention, such as a rollback or hotfix.

This is an illustrative contract, not a universal definition. A team using release trains, feature flags or multiple production regions will need to explain its choices.

Separate three kinds of missingness

No activity: the source is current and the service recorded no deployments in the window. Deployment frequency can accurately be zero. A deployment-based failure ratio is undefined because its denominator is zero.

No observation: the source is disconnected, stale or incompletely imported. You don't know the deployment count. Use "not measured", not zero.

Incomplete classification: deployments are observed, but incident links or failure labels are unresolved. You may know how much you deployed without knowing the change fail rate.

These states suggest different work. No activity may call for a conversation about batch size. No observation calls for a connection or data-quality fix. Incomplete classification calls for agreement on the failure workflow. A single green badge hides that distinction.

Put the counts beside the ratio

Consider an invented month with forty observed production deployments. Two required a rollback and one required a hotfix. Assume those are three distinct failed deployments, not two records describing the same failure.

The observed change fail rate is 3 / 40 = 7.5%.

Now suppose only thirty of the forty deployments were imported. Publishing 3 / 30 = 10% as the service's complete monthly rate would change the question without telling the reader. It describes the imported subset, not necessarily the service's full month.

A compact reporting line can make this visible:

Change fail rate: 7.5% (3 of 40 deployments). One service, September window. Deployment source current through October 1. Failure classification reviewed for this window.

If the import is incomplete, replace the confident headline:

Change fail rate: not finalized. Thirty deployments observed; source coverage for the reporting window is incomplete.

Use "reviewed" only when that review actually happened. Coverage is a separate measure from performance.

Don't manufacture a recovery-time average

Failed deployment recovery time concerns deployments that fail and require immediate intervention. A month without such an event has no recovery-time sample. Reporting "0 minutes" makes absence look like instant recovery.

Show the sample count and say "no qualifying events observed" when it is zero. If there are samples, document which timestamps mark failure and recovery. Excluding an unresolved incident can make a chart look better while leaving the hardest failure out of the calculation.

Run a ten-minute review before sharing

Ask the service owner and the person maintaining the measurement to check:

DORA cautions against making metrics the goal and against measuring at the expense of improvement. A trustworthy denominator is not the finish line. It makes the next conversation honest: what should this service change, and how will the team know whether it helped?

• Is the service and reporting window explicit?
• Are relevant source imports current for that whole window?
• Are duplicates, retries and rollbacks handled consistently?
• Does every ratio expose its numerator and denominator?
• Are no activity, no observation and incomplete classification distinct?
• Does every duration include its sample count and unresolved-event treatment?
• Is the next action about improving delivery, rather than winning a ranking?

Sources

DORA, "DORA's software delivery performance metrics", inspected October 13, 2026: https://dora.dev/guides/dora-metrics/

DORA, "How to transform", inspected October 13, 2026: https://dora.dev/guides/how-to-transform/

The reporting-state distinctions, checklist and invented arithmetic examples are editorial recommendations, not additional DORA standards. No ShipReady feature, customer result, benchmark, price or legal-readiness claim is made here.