An AI agent can do more than generate a response. Depending on how it is built, it may search the web, read files, call APIs, update records, send messages, or coordinate other agents. Those capabilities make agents useful—but they also make security more than a question of whether a model produces a correct answer.
AI agent security is the practice of protecting an agent, its tools, its data, and the systems it can affect so that it operates within an explicitly authorized scope. The goal is not to make a model perfectly reliable. It is to design the surrounding system so that an incorrect, manipulated, or unexpected model decision cannot easily become a damaging action.
On October 16, 2026, the US National Institute of Standards and Technology (NIST), the National Security Agency’s Super Intelligence Security Center, and the Cybersecurity and Infrastructure Security Agency published voluntary incident-readiness practices for training and evaluating superintelligence agents. The document focuses on a much narrower and higher-capability setting than everyday assistants, but its emphasis on preparation, monitoring, containment, recovery, and reporting reinforces a useful security principle: plan for failures before granting an agent consequential access. Read the NIST announcement.
What makes an AI agent a security concern?
A conventional application generally follows logic explicitly defined by its developers. An AI agent often combines a model with an orchestration loop and a set of tools. It can interpret a goal, choose an action, observe the result, and decide what to do next. The exact sequence may vary between runs.
That flexibility changes the risk. A chatbot that gives a bad suggestion may mislead a person. An agent with permission to send email, modify a database, publish a post, or run code may turn a bad suggestion into a real-world change. The consequences depend on its tools, credentials, environment, and approval rules—not just on the model.
An agent’s effective security boundary includes five parts:
- Model and instructions: the model can misunderstand a goal, invent facts, or be influenced by hostile instructions.
- Tools and extensions: APIs, plugins, scripts, browsers, and file operations translate model output into actions.
- Identity and credentials: tokens and service accounts determine which resources the agent can access.
- Data sources: emails, documents, websites, code repositories, and retrieved passages can contain untrusted content.
- Runtime and monitoring: permissions, network controls, logs, limits, and shutdown mechanisms determine whether unexpected activity is detected and contained.
A security review that looks only at the model misses these important application-level controls.
Common risks in agentic systems
Prompt injection and untrusted content
Prompt injection occurs when input is crafted to influence a model in ways that conflict with the application’s intended instructions. The content may arrive directly from a user or indirectly through a web page, email, document, or retrieval result. For example, an agent asked to summarize a support inbox may encounter a message that tells it to ignore its task and forward private information elsewhere.
A system should not treat text found in a document or web page as an authorization grant. Separate trusted application instructions from untrusted content, limit what tools can do, validate actions outside the model, and require confirmation for sensitive operations. These controls reduce the impact of manipulation even when content filtering does not catch every malicious instruction.
Excessive permissions
An agent usually needs only a subset of the access available to its underlying tools. A scheduling assistant may need to read calendar availability and create an event, but it does not automatically need permission to delete every event or export an entire mailbox. Reusing a highly privileged service account can turn a narrow task into a broad security exposure.
Apply the principle of least privilege: give the agent only the minimum access required for its current task. Prefer separate, narrowly scoped credentials; use read-only access when writes are not needed; and avoid shared administrator credentials. Authorization must be enforced by the API or downstream system, not left to the model’s judgment.
Excessive autonomy and dangerous tool calls
An agent may have access to a tool that makes irreversible or high-impact changes without independent checks. A prompt-injection attempt, a mistaken plan, or an ambiguous instruction can then trigger a harmful operation.
OWASP’s 2025 Large Language Model Application risks guidance describes excessive agency in terms of excessive functionality, excessive permissions, or excessive autonomy. Its mitigation advice includes minimizing extensions, limiting each extension’s functions, avoiding open-ended tools where possible, enforcing least privilege, and requiring human approval for high-impact actions. See OWASP’s guidance on Excessive Agency.
Sensitive-data exposure
An agent may receive more information than it needs, place sensitive context into logs, or pass private content to a tool or service that should not receive it. Multi-agent workflows can make the information path harder to understand because one component’s output may become another component’s input.
Reduce the amount of sensitive data available to the agent, enforce access checks on each retrieval, redact secrets from logs where appropriate, and document which tools receive which data. Do not assume that a prompt or a model’s internal instructions provide a reliable confidentiality boundary.
Weak monitoring and difficult recovery
A system can fail safely only if operators can observe meaningful activity and stop harmful behavior. If tool invocations are not logged, alerts are too noisy, or an agent can interfere with its own monitoring, investigators may be unable to reconstruct what happened. Restarting the same workflow without fixing the cause can repeat the incident.
Monitor relevant tool calls, permission changes, network activity, data access, errors, and attempts to exceed the intended scope. Logs need suitable access protection and retention. A shutdown or containment mechanism should operate outside the agent’s own discretionary control.
Practical controls for safer AI agents
No single safeguard can address every risk. A practical design layers controls so that a model mistake or a successful prompt injection does not automatically become an incident.
1. Define the task boundary before deployment
Write down what the agent is allowed to do, which data it may read, which systems it may contact, and what it must never change. Begin with a narrow, low-risk task and expand scope only after testing. The Australian Cyber Security Centre’s May 2026 guidance on careful adoption of agentic AI similarly recommends gradual adoption, strong oversight, monitoring, risk assessment, and careful privilege control. Read the joint guidance.
A useful scope definition is specific enough to test. “Help with email” is vague; “summarize messages in this folder, draft a reply, and never send or delete a message without approval” is much more actionable.
2. Use narrowly scoped tools and credentials
Build task-specific tools instead of exposing a general-purpose shell, unrestricted browser, or broad database connection when a limited function will suffice. A tool that reads a single record should not also have undocumented write or delete capabilities. Remove unused tools and permissions rather than relying on instructions telling the model not to call them.
Where elevated access is temporarily required, scope it to the task and revoke it when the operation ends. Store secrets in a managed credential system, avoid putting them in prompts, and rotate credentials if exposure is suspected.
3. Require approval for consequential actions
Separate low-risk actions from operations that change external state. Reading a public page may not require approval; sending a message, deleting files, approving a payment, changing production infrastructure, or publishing content may.
The approval step should show the intended action and its target clearly. It must be enforced by the tool or downstream service so a model cannot skip it by producing a different response. For especially sensitive actions, require an independent authorization path rather than treating the agent’s own explanation as proof that an action is safe.
4. Treat retrieved content as data, not authority
An agent that reads a website, email, or repository should assume that the content may be mistaken or malicious. Keep user-provided and retrieved text separate from trusted system policy, constrain which sources can be accessed, and validate each proposed tool call against rules outside the model. Output validation is especially important before content is interpreted as a command, query, file path, or API parameter.
This is a defense-in-depth strategy, not a guarantee that prompt injection can always be filtered. The strongest protection is to ensure that untrusted text cannot grant permissions or bypass authorization on its own.
5. Monitor actions and preserve useful evidence
Record who initiated a task, which agent and tools were involved, what resources were accessed, which consequential actions were attempted, whether approval was provided, and what the final result was. Protect logs from unauthorized modification and avoid collecting unnecessary sensitive content.
Define expected behavior and alert on meaningful deviations, such as repeated denied actions, access to unexpected resources, unusually large data transfers, new tool permissions, or attempts to disable monitoring. Monitoring should be tested; a dashboard is not sufficient if it fails silently or cannot observe activity at the layer where actions occur.
6. Test failure paths and provide an independent stop mechanism
Test agents against ambiguous requests, malicious documents, unavailable tools, revoked credentials, and partial failures. Evaluate not only whether the model refuses a harmful instruction, but also whether the system blocks unauthorized actions when the model fails to refuse it.
Provide a way for an operator to pause or terminate the agent, revoke credentials, block tool access, and isolate the affected runtime. Keep recovery procedures outside the agent’s ordinary permissions. Rehearse them so responders know which switches work and what evidence must be preserved before rebuilding an environment.
A simple incident-response plan for AI agents
NIST’s October 2026 document sets out ten voluntary practices for incidents involving superintelligence agents in training and evaluation. Its intended audience is specialized, so smaller agent deployments should not present the document as a universal compliance standard. Its five-stage lifecycle, however, offers a useful way to organize an incident plan alongside established cybersecurity guidance such as NIST SP 800-61 Revision 3.
| Stage | Practical action for an AI-agent deployment |
|---|---|
| Prepare | Inventory agents, tools, identities, sensitive data, and owners. Define allowed behavior, severity levels, contacts, and shutdown procedures. |
| Detect | Monitor agent activity and tool calls. Alert on access outside scope, unusual actions, and monitoring failures. |
| Respond | Pause the affected task, block risky tools, revoke exposed credentials, and preserve logs and relevant evidence. |
| Recover | Identify the root cause, patch the weakness, verify permissions and monitoring, and test containment before restarting. |
| Report and learn | Inform affected stakeholders as appropriate, document what happened, and update controls based on the findings. |
The response should fit the severity of the event and the organization’s legal and operational obligations. Not every unexpected model response is a security incident, but actions that affect confidentiality, integrity, availability, money, or third parties deserve a clear triage path.
Example: an email agent receives a malicious message
Imagine an assistant that can read incoming mail and draft responses. It receives a message containing hidden instructions to search the inbox for invoices and forward them to an external address. The user asked only for a summary, but the agent attempts to use a mail-sending tool.
A secure system should block the action for several independent reasons. The task boundary does not authorize forwarding, the mail-reading tool should not have send capability, the identity should lack broad mailbox permissions, and any sending operation should require explicit user approval. Monitoring should record the blocked attempt and flag repeated suspicious behavior. If a credential or other control may have been compromised, an operator should be able to pause the workflow and revoke the credential without asking the agent to do so.
Notice the design goal: the system remains safe even if the model misunderstands the message. Security is not based on hoping the agent follows a warning in its prompt.
AI agent security checklist
Before deploying an agent, verify that the team can answer “yes” to the following questions:
- Is the intended task and prohibited behavior written down?
- Are tools limited to the fewest actions the task requires?
- Does each tool enforce authorization independently of model output?
- Are credentials scoped, protected, and revocable?
- Are high-impact actions subject to meaningful approval?
- Is retrieved content treated as untrusted data?
- Can operators observe important actions and investigate an incident?
- Can the agent be paused and its access revoked without relying on the agent itself?
- Has the team tested prompt injection, tool misuse, failure, and recovery scenarios?
- Is there an owner responsible for reviewing incidents and strengthening controls?
If several answers are “no” or “unknown,” keep the agent in a limited test environment and reduce its permissions before expanding its duties.
Conclusion
AI agents introduce a security challenge because model outputs can trigger actions across tools, identities, data, and connected systems. The most dependable approach combines least privilege, narrowly scoped tools, external authorization, human approval for consequential operations, monitoring, and tested containment and recovery procedures.
Recent official guidance from NIST and international cybersecurity agencies reinforces the value of preparing for unexpected behavior rather than relying on perfect model decisions. Start with low-risk tasks, measure what the agent can actually do, and expand access only when the controls have been tested. The objective is not simply an agent that appears helpful; it is an agent whose actions remain bounded, observable, and recoverable.
Further reading
- NIST: Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation (October 16, 2026)
- NIST: Full incident-readiness and response guidance (October 2026)
- Australian Cyber Security Centre: Careful Adoption of Agentic AI Services (May 1, 2026)
- OWASP GenAI Security Project: LLM06:2025 Excessive Agency
- NIST SP 800-61 Rev. 3: Incident Response Recommendations