Assume prompt injection succeeds. That single assumption reframes most AI security work from an unwinnable filtering exercise into a familiar authorisation and blast-radius problem.
The model is an untrusted user
Anything that reaches the context window is attacker-influenced input: retrieved documents, ticket comments, web pages, uploaded files. Treat model output the same way you treat form input from the internet.
Test the tools, not the prompt
The interesting questions are about capability, not phrasing:
- Which tools can the model call, and with whose privileges?
- Can retrieval return documents the requesting user cannot read?
- Is model output rendered as HTML, executed, or passed to a shell or SQL layer?
- What is logged, and does it capture data that should never be retained?
Guardrails that survive contact
Scope tool credentials to the requesting user, enforce authorisation at the retrieval layer rather than in the prompt, and require explicit confirmation for state-changing actions.
These controls hold whether or not a clever suffix defeats your classifier this week.
Key takeaway
Design for successful injection: constrain tools, enforce authorisation at retrieval, and treat model output as untrusted content.