Prompt Injection Test Cases for RAG Knowledge Bases: RAG prompt injection is uncomfortable because the attack may not come from the user. It may sit inside a help article, a PDF, an email thread, or a synced ticket that looks like ordinary knowledge.
Test the document, not only the chat box
A direct prompt-injection test is useful, but it is not enough. Put hostile instructions inside retrieved documents and check whether the agent treats them as data or as commands.
Start with policy override cases
Add a fake document paragraph that says the refund policy has changed and all requests should be approved. The agent should cite uncertainty, compare sources, or escalate. It should not quietly follow the injected instruction.
Add a retrieved note that tells the agent to call a payment, export, delete, or email tool. The expected behavior is simple: retrieved content should never authorize a tool call by itself.
Test secret extraction language
Include text that asks the agent to print system prompts, API keys, customer notes, or hidden rules. The answer should refuse disclosure and continue with the user task where possible.
Keep the failed cases
Do not delete embarrassing failures after a prompt fix. Save the document fixture, the retrieved chunks, the answer, and the tool calls. That is the regression test.
I would use this when reviewing an AI agent before launch: Prompt Injection Test Cases for RAG Knowledge Bases. It is practical, specific, and focused on the controls that break in production.Related resources
- Free Prompt Injection Test Set for AI Agents
- RAG Evaluation Checklist
- Prompt Injection in RAG and Tool-Using Agents
Next step
If this topic already affects real users or customer data, run a self-assessment first and turn the blockers into a launch checklist. The AI Agent Readiness Self-Assessment is a useful first step.