The pipeline should provide layered confidence
Code Commit
↓
Build + Unit
↓
Pull Request Checks
↓
API / Contract / Component
↓
Deploy to Test
↓
Smoke + Integration
↓
Broader Regression
↓
Release Candidate
↓
Production Validation
Pull request checks
PR feedback should be fast enough to support developer flow. Prioritize unit, component, linting, static analysis, selected API/contract checks, and high-value targeted regression.
Post-deployment checks
After deployment, validate environment health, connectivity, configuration, identity, feature flags, and critical smoke flows before broader execution.
Nightly and scheduled suites
Long-running combinations, broader regression, browser matrices, data-heavy validation, and selected performance tests may run on a schedule rather than on every commit.
Release-candidate validation
- Critical regression
- Open defects assessed
- Deployment rehearsal
- Rollback readiness
- Monitoring ready
- Data migration validated
- Operational support ready
- Residual risk visible
Do not put every test in every stage
Pipeline design is an optimization problem: maximize useful confidence while preserving fast feedback.
Quarantine carefully
Flaky tests may be temporarily quarantined, but quarantine must be visible, owned, and time-bounded. Otherwise the suite gradually loses protection.