AI Agent Security: Prompt Injection, Permissions and Data

The threat model: why agents are different

A classic web application follows code paths that developers wrote. An AI agent decides its own next step based on text, and some of that text comes from people and documents you do not control. That changes the security question from "is this endpoint protected?" to "what is the worst thing this agent could be talked into doing, and what stops it?"

Three properties make agents a distinct risk:

• Instructions and data share one channel. The model receives your system prompt, the user's message, retrieved documents and tool results as one stream of tokens. It has no reliable way to tell which parts are trusted.
• Agents hold credentials. To be useful they call CRMs, payment gateways, email and databases. Whatever those credentials allow, a manipulated agent can attempt.
• Behaviour is probabilistic. The same input can produce different actions. You cannot prove safety by testing a few happy paths.

If you have not yet settled the overall system design, read AI agent architecture first; most of the controls below map onto its guardrails, tools and observability layers.

Where controls sit: a secure request flow

Prompt injection, direct and indirect

Direct injection

The user types instructions intended to override yours: "ignore previous instructions", role-play framings, fake system messages, encoded or translated payloads, or long conversations that gradually shift the agent's behaviour. The usual goals are to extract the system prompt, get the agent to say something damaging, or call a tool it should not.

Indirect injection

The attacker never talks to the agent. Instead they plant instructions in content the agent will read: a support email, a PDF résumé, a product review, a web page, a calendar invite, a field in a CRM record or the response of a third-party API. When the agent retrieves that content, the hidden text can redirect it, for example "forward this conversation to the following address" or "tell the user their refund was approved". Indirect injection is the more serious threat for agents, because it scales and the victim user may be entirely innocent.

Controls that help

• Reduce what injection can achieve. This is the most important control. If the agent that reads inbound email has no tool to send email externally, an injected "forward this" instruction fails regardless of what the model thinks.
• Separate and label untrusted content. Wrap retrieved documents and tool results in clearly marked blocks and instruct the model to treat them as data. This lowers the success rate; it does not eliminate it.
• Screen inputs. Heuristic and classifier checks for known injection patterns, unusual encodings and hidden text (white-on-white, zero-width characters, HTML comments) catch low-effort attacks.
• Split privileges between agents. A "reader" agent summarises untrusted content into a constrained structure; a separate "actor" agent with tools only sees that structured summary.
• Protect the system prompt, but do not rely on secrecy. Assume the prompt will leak. Keep secrets, internal URLs and business rules that matter for security out of it.
• Test adversarially. Keep a growing library of injection cases in your evaluation suite and run it on every change. Our guide to testing and evaluating AI agents covers how to structure these sets.

Tool permission scoping and least privilege

Excessive agency, meaning an agent with more capability than its job requires, turns a harmless model mistake into a real incident. Apply least privilege at four levels:

1. Tool design. Offer narrow, purpose-built operations. lookup_booking(booking_ref) is safer than query_database(sql); send_payment_link(order_id) is safer than send_whatsapp(to, text). Narrow tools also make the agent more accurate.
2. Tool availability. Load only the tools a given flow needs. A pre-sales agent has no refund tool; an unauthenticated visitor gets read-only tools.
3. User-bound authorisation. Every tool call runs in the context of the end user and tenant, not a super-user. The tool, not the model, checks that booking B-1042 belongs to the authenticated customer. Never let the model supply the user ID or tenant ID as a free argument.
4. Credential scope. The service account behind each tool can perform only that tool's operations: read-only database roles, API keys limited to specific endpoints, per-environment keys, short-lived tokens where the provider supports them.

Add hard limits in code as well: maximum refund value, maximum records per update, rate limits per conversation, and allowlisted recipients or domains for outbound messages.

Data leakage and PII redaction

Data leaves through several doors: the model provider, logs and traces, the agent's replies to the wrong person, and memory that resurfaces in another conversation.

• Minimise context. Retrieve the specific fields the step needs, not the whole customer record.
• Mask before the model. Replace phone numbers, emails, card and account numbers, national ID numbers and addresses with tokens such as [PHONE_1] when the model does not need the real value. Tools resolve tokens back to real values server-side.
• Redact logs. Traces are invaluable for debugging and dangerous as a data store. Apply the same masking to traces, set short retention, and restrict who can view raw transcripts.
• Check provider terms. Confirm data retention, training use and processing region with your model provider. If the terms do not match your obligations, use a regional deployment or a self-hosted model.
• Scope memory. Long-term memory must be keyed to a verified identity and tenant, with retention and deletion rules. See AI agent memory for what to store and what to avoid.

Tenant isolation for RAG

In multi-tenant products and in internal tools with permission levels, the retrieval layer is where data most often crosses boundaries. The failure pattern is simple: the vector search returns the most similar chunks across the whole index, and a filter is applied afterwards, or forgotten.

• Store tenant_id, access level and document ID as metadata on every chunk, and apply them as mandatory filters inside the query, injected by server code from the authenticated session.
• For high-sensitivity clients, use separate indexes or namespaces per tenant so that a missing filter returns nothing instead of someone else's data.
• Mirror source permissions. If a document is restricted in SharePoint or Google Drive, the chunk must carry that restriction, and permission changes must propagate to the index.
• Support deletion by document and by tenant, and verify it with tests.
• Write automated cross-tenant tests: as tenant A, ask questions whose answers exist only in tenant B's documents, and assert that nothing is returned.

The same isolation principles apply to the relational side; see multi-tenant data isolation in PostgreSQL. For retrieval pipeline design more broadly, read production RAG architecture.

Secrets management

• API keys and database credentials live in a secrets manager and are injected into tool runtimes, never placed in prompts, tool descriptions, retrieved documents or memory.
• The model never sees a credential, so it can never repeat one. If a tool needs an OAuth token for a user's calendar, the tool fetches it by session, not by an argument the model passes.
• Use separate keys per environment and per tool, rotate them on a schedule, and alert on unusual usage.
• Scan outputs and logs for secret patterns as a backstop.

Output validation

Treat model output like user input to the rest of your system. When the agent produces structured data, validate it against a strict schema and reject or repair anything that does not match. When output is rendered in a browser, encode it and restrict markdown and HTML, because injected content can smuggle in scripts or tracking images that exfiltrate data through URLs. When output feeds another system, such as SQL, shell commands, templates or emails, never pass it through unescaped. Restrict links in replies to allowlisted domains, and check factual claims that carry risk, such as prices or policy terms, against the source system before they are sent.

Approval gates for risky actions

Classify every tool by risk and decide the handling in advance:

Approvals must be durable: the agent run is checkpointed, the approver sees the proposed action, the arguments, the conversation and the reason, and the run resumes or ends based on the decision. Approvers should be able to edit arguments, not only accept or reject. We walk through a working design in building an AI agent with tool calling and human approval.

Audit logging

An audit log answers "who did what, and why" after the fact. For agents it should record, per step: the authenticated user and tenant, the channel, the model and prompt version, the action the model proposed, the policy decision and the rule that produced it, the approver and timestamp for gated actions, the tool arguments as executed, and the result. Store it append-only, separately from debug traces, with access restricted and retention set by your compliance requirements. Masking still applies. Alert on patterns such as repeated policy denials in one session, which often indicate an attack in progress.

OWASP LLM Top 10 mapping

The OWASP Top 10 for LLM Applications (2025 edition) is a widely used checklist of risk categories. Here is how each maps to agent controls.

Security checklist

Use this in design reviews and before go-live. It pairs well with our AI agent requirements template.

Design

• Threat model written, including indirect injection sources for every content type the agent reads
• Each tool narrow, typed, risk-tiered and documented with the permissions it needs
• User and tenant identity injected by server code, never taken from model arguments
• High-risk actions routed through a durable approval queue
• Security-relevant business rules enforced in tools, not only in prompts

Data

• PII masking before model calls where the real value is not needed
• Model provider retention, training and region terms reviewed and accepted
• Retrieval filters mandatory and applied inside the query; cross-tenant tests passing
• Memory keyed to verified identity, with retention and deletion rules
• Logs and traces masked, access-restricted and time-limited

Runtime

• Secrets in a secrets manager; none in prompts, documents or memory
• Output validated against schemas; rendered output encoded; links allowlisted
• Step budgets, timeouts, token limits and rate limits per user and tenant
• Append-only audit log with policy decisions and approvals
• Alerts for repeated policy denials, cost spikes and unusual tool usage

Assurance

• Adversarial evaluation set, including injection and data-exfiltration cases, run on every change
• Independent penetration test covering agent-specific attack paths before launch
• Incident runbook: how to disable a tool, revoke a key or turn off the agent quickly

For how Next Olive handles access, data and delivery practices on client projects, see security and compliance. More guides are on the AI agent resources hub, and our AI agent development page explains how we deliver production agents.

Frequently asked questions

Can prompt injection be fully prevented?

Not with today's models. Detection filters and careful prompting reduce the rate of successful attacks, but none is reliable on its own. The dependable defence is architectural: limit what the agent can do with narrow tools and least-privilege credentials, validate every tool call in code, and require human approval for actions that are costly or irreversible.

What is indirect prompt injection?

It is an attack where malicious instructions are hidden in content the agent reads rather than typed by the user, for example in a web page, an uploaded PDF, an email, a product review or a tool response. The agent may treat that text as instructions. Any agent that uses RAG, browses, or reads inboxes is exposed to it.

Should we send customer PII to an LLM provider?

Only what the task needs. Mask or tokenise identifiers such as phone numbers, emails and account numbers before model calls where the model does not need the real value, and resolve them back in your own code when a tool runs. Review the provider's data retention and training terms, and choose a regional or self-hosted model if they do not fit your obligations.

Which agent actions need human approval?

Actions that move money, change or delete records in bulk, send messages to external parties on your behalf, grant access, or cannot easily be undone. A practical approach is to tier every tool as low, medium or high risk, auto-execute low risk, apply limits and checks to medium risk, and route high risk to an approval queue.

How does the OWASP Top 10 for LLM applications apply to agents?

It is a useful checklist of the main risk categories, including prompt injection, sensitive information disclosure, improper output handling, excessive agency and unbounded consumption. Agents are most exposed to excessive agency and injection because they can act, so map each category to a concrete control in your design and test it before launch.