Metrics should answer a decision question
Good quality metrics help leaders understand risk, capability, flow, and improvement. They should not exist simply because a tool can count something.
AskAQA principle: Measure outcomes and decision signals—not activity for its own sake.
Useful metric families
| Family | Examples |
|---|---|
| Coverage | Requirement, risk, P1/P2 regression coverage |
| Defects | Severity mix, leakage, aging, reopen rate |
| Automation | Critical regression automated, flake rate, duration |
| Execution | Pass/fail, blocked tests, cycle time |
| Release | Open risk, readiness status, post-release defects |
| Maturity | Capability scores, adoption, improvement completion |
Avoid vanity metrics
- Total test case count
- Total bugs found without severity/context
- Automation percentage without denominator
- Pass rate without coverage
- Lines of test code
Build executive and team views differently
Executives need trends, risk, and outcomes. Teams need actionable detail: failures, flaky tests, coverage gaps, aging defects, and environment issues.
Show trends, not snapshots
A single number can mislead. Trend escaped defects, automation health, coverage, and maturity over time.
Automate dashboard data
Where possible, pull metrics directly from source systems. Manual reporting is slow, error-prone, and difficult to sustain.