What is a flaky test?
A flaky test produces inconsistent outcomes without a relevant product change. It passes, fails, then passes again under apparently equivalent conditions.
Rerun-until-green is not a reliability strategy. It hides signal quality problems.
Common causes
| Cause | Example |
|---|---|
| Timing | Test acts before async work completes. |
| Shared state | Tests modify the same account. |
| Selectors | Locator depends on changing DOM structure. |
| Data collision | Parallel tests reuse identifiers. |
| Environment | Service is intermittently unavailable. |
| Order dependency | Test passes only after another test. |
Replace sleeps with conditions
Fragile
Wait 5 seconds and hope.
Reliable
Wait for the specific observable state required by the next action.
Classify failures
Automation Failure
├── Product defect
├── Test defect
├── Test data issue
├── Environment issue
├── Dependency issue
└── Expected product change
Treat flakiness as technical debt
Track flake rate, assign ownership, repair high-value flaky tests quickly, and quarantine only temporarily.
If a test needs repeated retries to be trusted, the product, environment, data setup, or test code needs engineering attention.