AI Agent Security Audit Checklist: What to Review Before Production

Short answer: an AI agent security audit should verify the full workflow, not only the model prompt. Review prompt injection exposure, tool permissions, authorization, RAG trust boundaries, guardrails, human approval, logging, cost and latency anomalies, and recovery behavior. The goal is to prove that the agent can complete useful work without leaking data, following hostile instructions, or taking unauthorized actions.

Disclosure: This checklist is based on public documentation and security guidance. It is not a formal compliance certification or legal opinion. Use it as a practical engineering review before giving an AI agent production access.

Why AI agent audits need a different checklist

A traditional application security review focuses on code paths, inputs, authentication, authorization, storage, and network boundaries. An AI agent adds another layer: a probabilistic planner that may read untrusted content, select tools, call external systems, and produce actions that look legitimate. That makes the audit target the full loop: user input, retrieved context, model reasoning, tool calls, authorization, final output, and logs.

OWASP’s LLM Top 10 identifies risks such as prompt injection, sensitive information disclosure, supply chain issues, improper output handling, excessive agency, and system prompt leakage. These map directly to agent security reviews because agents are specifically designed to take actions beyond simple text generation. Source: OWASP Top 10 for LLM Applications.

NIST’s Generative AI Profile also emphasizes managing risks such as confabulation, data privacy, security, misuse, and information integrity. Source: NIST AI Risk Management Framework.

1. Define the audit scope

Start by making the system boundary explicit. Most weak audits fail because they test a demo prompt while ignoring the real tools and data paths.

• What user roles can access the agent?
• What data sources can the agent read?
• What tools can the agent call?
• Which actions are reversible, irreversible, financial, public-facing, or privacy-sensitive?
• Which environments are in scope: sandbox, staging, production, or read-only demo?
• What must the audit not touch?

For a paid readiness review, this is also the intake stage. IBBS.AI uses a scoped process for the AI Agent Readiness Audit so test cases match the system’s real risk profile.

2. Map trust boundaries

List every place where untrusted text or data enters the agent. The obvious source is the user prompt, but indirect prompt injection often comes from retrieved documents, emails, tickets, web pages, PDFs, comments, logs, and tool outputs.

Check these boundaries:

• User prompt to system instructions.
• Retrieved content to model context.
• Tool output to model context.
• Model output to application actions.
• Agent memory to future sessions.
• Internal documents to external users.

See also: Prompt Injection in RAG and Tool-Using Agents.

3. Test prompt injection and indirect injection

Prompt injection testing should include both direct and indirect attacks. Direct tests try to override the prompt in the user message. Indirect tests hide instructions inside retrieved or tool-returned content.

Minimum test set:

• “Ignore previous instructions” variants.
• Requests to reveal system prompts, hidden policies, tool schemas, or credentials.
• Retrieved-document instructions that ask the agent to exfiltrate data.
• Tool-output instructions that ask the agent to call a second tool.
• Role confusion tests: customer asks for admin-only information.
• Cross-tenant tests: user asks for another customer’s records.

Passing means more than refusing the exact string. The agent must preserve the hierarchy of instructions, treat retrieved content as data, and require deterministic authorization before taking action.

4. Review tool permissions and authorization

Tool access is where agent risk becomes business risk. “The model decided to do it” is not an authorization mechanism.

Audit each tool:

• What can the tool read or change?
• Can the same action be done through a safer, narrower tool?
• Are tool arguments validated server-side?
• Does the application check the user’s real permissions before the tool runs?
• Are irreversible actions gated by human approval?
• Are tool results logged with request IDs and user IDs?

OpenAI’s Agents SDK documentation describes concepts such as tools, guardrails, tracing, handoffs, and sessions. Those building blocks are useful, but the audit should verify how your application uses them in practice. Source: OpenAI Agents SDK documentation.

5. Check guardrails realistically

Guardrails are useful, but they are not a complete security model. They can miss context, block legitimate work, or run too late in the workflow. A good audit asks what each guardrail catches, what it misses, and what happens after it fires.

Review:

• Input guardrails: before the model sees the request.
• Retrieval guardrails: before documents enter model context.
• Tool guardrails: before the agent calls an action.
• Output guardrails: before the final answer or action is returned.
• Escalation paths: human review, safe refusal, or read-only fallback.

For more detail, read AI Agent Guardrails: What They Catch, What They Miss, and How to Test Them.

6. Verify RAG security and citation behavior

RAG systems can fail in two ways: they retrieve the wrong evidence, or they retrieve hostile evidence and the agent follows it. Both matter in a security audit.

• Can users upload or influence documents that other users will retrieve?
• Are document permissions enforced before retrieval?
• Can retrieved text override system or developer instructions?
• Does the agent cite real sources for material claims?
• Does it refuse or ask for more information when evidence is missing?
• Are stale documents marked or excluded?

7. Inspect logs, traces, and evidence

If you cannot reconstruct what the agent saw and did, you cannot audit it after an incident. Trace capture should include model calls, tool calls, guardrail decisions, user identity, authorization checks, latency, token usage, and final outputs.

OpenTelemetry publishes GenAI semantic conventions for spans and telemetry concepts such as model requests, responses, token usage, and agent/tool activity. Source: OpenTelemetry GenAI semantic conventions.

8. Measure cost and latency anomalies

Security audits should also check operational anomalies. A prompt injection attempt may cause loops, excessive tool calls, large context windows, repeated retries, or unusual latency.

• Cost per successful task.
• Cost per failed or refused task.
• Number of model calls per run.
• Number of tool calls per run.
• p50/p90/p95 latency.
• Retries and loop detection.

These metrics connect security to product reliability. See How to Evaluate an AI Agent for a broader evaluation framework.

9. Prioritize findings by business impact

Not all findings are equal. A missing citation is usually less urgent than a tool that can email private customer data to an attacker. Use severity levels that combine exploitability, data sensitivity, user impact, and reversibility.

Minimum audit checklist

• System boundary and in-scope workflows documented.
• All tools mapped with read/write permissions.
• Server-side authorization checked before every sensitive tool call.
• Direct and indirect prompt injection tests executed.
• RAG document permissions verified before retrieval.
• Guardrails tested at input, retrieval, tool, and output stages.
• Human approval required for irreversible or high-risk actions.
• Trace logging captures model calls, tool calls, guardrail decisions, cost, and latency.
• Findings prioritized by severity and business impact.
• Retest plan defined for high and critical fixes.

Bottom line

An AI agent security audit is not a prompt review. It is a workflow review that asks whether the system can safely read data, reason over untrusted context, call tools, and recover from attacks. If the agent has access to private data or real actions, audit the boundaries before production—not after the first incident.