Prompt injection in multi-agent systems
Black Hat: prompt injection on multi-agent LLM systems bounded by agent permissions Black HatTL;DW
- Prompt injection power is bounded by agent permissions—control planner output controls plans; control tool-use agent controls tool execution.
- Observability is critical: collect telemetry at LLM-to-code seams (where system prompt meets dynamic content) to detect attacks early.
- Mirror system prompt patterns (markdown, spacing, tool argument names) when crafting prompt injections for higher success rates.
- Data exfiltration often requires chaining LLM compromise with infrastructure hacks (CSP bypasses, expired domain purchases, credential misuse).
- Stored prompt injections via RAG documents can persistently infect user long-term memory, enabling lateral platform attacks across multiple users.
- Use lightweight prompt guards (Purple Llama's 300M-parameter model on CPU) for fast detection on dynamic content only, not full prompts.
- Enforce tool-call policies: orchestrators must validate that agents call tools in standard ways with correct arguments and permissions.
- LLM-as-judge with few-shot examples of platform-specific prompt injections generates medium-to-weak detection signal when deployed in parallel.
- Scope agent capabilities per task: grant minimal permissions for each session, revoke after completion to limit blast radius.
- Attacks are non-deterministic—prompt injections failing initially doesn't mean success is impossible; attackers retry dozens to hundreds of times.
TL;DW
- Prompt injection power is bounded by agent permissions—control planner output controls plans; control tool-use agent controls tool execution.
- Observability is critical: collect telemetry at LLM-to-code seams (where system prompt meets dynamic content) to detect attacks early.
- Mirror system prompt patterns (markdown, spacing, tool argument names) when crafting prompt injections for higher success rates.
- Data exfiltration often requires chaining LLM compromise with infrastructure hacks (CSP bypasses, expired domain purchases, credential misuse).
- Stored prompt injections via RAG documents can persistently infect user long-term memory, enabling lateral platform attacks across multiple users.
- Use lightweight prompt guards (Purple Llama's 300M-parameter model on CPU) for fast detection on dynamic content only, not full prompts.
- Enforce tool-call policies: orchestrators must validate that agents call tools in standard ways with correct arguments and permissions.
- LLM-as-judge with few-shot examples of platform-specific prompt injections generates medium-to-weak detection signal when deployed in parallel.
- Scope agent capabilities per task: grant minimal permissions for each session, revoke after completion to limit blast radius.
- Attacks are non-deterministic—prompt injections failing initially doesn't mean success is impossible; attackers retry dozens to hundreds of times.
Maps the attack surface across orchestration frameworks with five CVEs—VS Code Copilot, Outlook Copilot, Salesforce agents—showing kill chains from RAG poisoning to CSP-bypass exfiltration. Defenses focus on context firewalls, scoped per-session capabilities, and telemetry at LLM-to-code boundaries.