AI Agent Synthetic Monitoring Checklist
A synthetic monitoring checklist for AI agents covering critical journeys, expected outcomes, tool checks, test accounts, and alerts.
AI Evaluation
Practical evaluation, reliability, and measurement methods for AI agents and LLM applications.
AI Agent Cost Regression Testing
A cost regression testing guide for AI agents covering per-task cost, baselines, retry loops, model routing, and budget gates.
AI Agent Post-Launch Review
A post-launch review checklist for AI agents covering production behavior, failures, escalations, costs, monitoring, and next actions.
AI Agent Evaluation Scorecard
A practical AI agent evaluation scorecard for scoring task success, safety, tool use, latency, cost, and launch blockers.
AI Agent Rollback Drill
A rollback drill guide for AI agents covering failure scenarios, rollback units, timing, evidence, and runbook updates.
AI Agent Release Gate Checklist
A release gate checklist for AI agent changes covering scope, tests, blockers, monitoring, owners, and release evidence.
AI Agent Data Retention Policy
A practical AI agent data retention policy checklist covering stored data, risk classification, retention periods, deletion workflows, and vendor review.
AI Agent Tool Failure Handling
How production AI agents should handle tool failures, retries, fallback behavior, logs, unsafe chains, and pre-launch failure testing.
AI Agent Hallucination Monitoring
A monitoring checklist for AI agent hallucinations covering evidence grounding, risky patterns, human review, regression tests, and answer constraints.
AI Agent Product Requirements Checklist
A product requirements checklist for AI agents covering users, allowed actions, knowledge sources, safety requirements, evaluation, and ownership.