Why agents are harder to test
AI agents do more than generate text. They may plan, select tools, call APIs, use memory, retry actions, and change external state.
Agent workflow model
Test tool selection
Confirm the agent chooses an appropriate tool and does not use privileged or irrelevant tools when a safer alternative exists.
Test tool parameters
A correct tool with incorrect parameters can still cause harm. Validate identifiers, dates, amounts, recipients, filters, permissions, and defaults.
Test failure recovery
- Tool timeout
- Rate limit
- Invalid response
- Partial failure
- Authentication failure
- Unavailable dependency
- Conflicting state
- Retry exhaustion
Test permission and approval boundaries
High-impact actions may require explicit human approval. Validate the agent cannot bypass approval logic through prompt wording, retry, memory, or tool chaining.
Test loops and runaway behaviour
Agents should have bounded retries, timeouts, action limits, and stop conditions. Include tests for repeated failed plans or circular tool use.
Test memory and state
Verify the agent uses relevant memory without leaking information between users, sessions, tenants, or unrelated tasks.
Evaluate final outcomes, not only steps
An agent can follow plausible steps and still fail the user's goal. Evaluate correctness, completeness, efficiency, safety, and side effects at the workflow level.