AI agents and the security problems we already know

AI agent security, part 1 of 4. An agent’s own credentials do not show every action it can request through other agents. How we review delegation, trust and open-source security coverage.

Companies are moving more work to AI agents, including work inside security teams. We have seen versions of the security failures around them for years. An account has more access than its job requires, or a service accepts a request because it came from somewhere internal. An agent can carry one task through several of these relationships, sometimes without anyone checking the original authorization again.

In McKinsey’s 2026 survey, 40% of respondents from companies with more than $1 billion in annual revenue said they were scaling agents in at least one business function. Microsoft already offers an agent that investigates reported phishing emails. McKinsey’s survey, Microsoft’s documentation

For our open-source projects, we have been reviewing what we can prevent, what we can detect, and where the gaps remain. OWASP’s Top 10 for Agentic Applications gives us a reference for that review, including privilege abuse, tool misuse, memory poisoning and insecure communication between agents. We use these risks to review our own projects in part two, including the controls we have implemented and the gaps they leave.

Permissions and delegation

We might give a support agent read-only access to tickets and no production credentials. That looks like a reasonable restriction. But the agent can ask an operations agent for help, and the operations agent can pass work to a deployment agent.

The deployment agent receives a request from another agent it recognizes. It has the credentials to make the change. Unless it checks whether the original task authorized that change, a read-only support investigation can end with a production deployment. The support agent never needed those credentials, or even a direct connection to the agent that used them.

The task needs to carry its authorization through these handoffs. Where agents simply accept each other’s requests and use whatever access they have, the first agent may be able to use much of the group’s combined capability. Reviewing its own account will miss that access. The failure is in accepting delegated work without the right authorization; allowing agents to communicate does not, by itself, give them each other’s permissions.

Checking the original requester is only part of this. A release task might allow production changes, then delegate a narrower job that permits staging only. A specialist receiving that job must keep the staging restriction, even if the original requester and the specialist’s service account both have production access. We need to preserve restrictions introduced at every handoff. Going back to the most privileged identity in the chain would lose them.

Anthropic describes a related issue in Claude Code auto mode. Inside a subagent, the orchestrator’s instruction appears as a user message. A safety check without the original context may see a task that looks authorized, even though another agent decided to do it. Anthropic checks the outgoing delegation while it can still distinguish that agent decision from the user’s request. Anthropic’s explanation

This is familiar territory for security teams. A privileged service acts for someone who was not entitled to the operation: the confused-deputy problem. Across agents, those trust relationships can provide a route for lateral movement. A workflow that describes which agent handles deployments or can access a database also supplies information about where to send the request. Some of the discovery work is already done.

For the production change, Open Policy Agent can evaluate whether the task has the required permission. The receiving service must enforce that decision using verified identity and authorization information. Another agent’s message saying “the user approved this” cannot supply the evidence.

Visibility across the whole task

The deployment log may show a valid account making a normal change. To see the problem, we have to follow that request back to the support task, which was only allowed to read. Looking at the final action in isolation can hide the reason it should have been refused.

We need to understand the available delegation paths before execution, then follow the path actually taken. That means preserving the original requester and task scope as work moves between agents, and connecting any approval to the resulting action. A record that ends when one agent hands work to another leaves us unable to explain what happened next.

OpenTelemetry can connect instrumented activity across services. We still have to record the relevant security decisions; a trace ID is not proof of permission. The agents should also be unable to rewrite the evidence. NIST’s proposed agent identity design tenets call for linked delegation, authority that cannot grow merely because work is passed downstream, and independent records of the actions taken. NIST’s guidance

The useful record connects the final operation to the permissions that survived each handoff. It should explain a refusal even when the worker at the end normally has production access. It should also let us distinguish that refusal from a tool that was simply broken.

Prompt injection and memory

An agent investigating an email or reading a repository has to process content written by someone else. That person can put instructions into the content too. The agent may then follow them using tools intended for the legitimate task.

We should be careful about carrying the wrong lesson over from older injection attacks. Parameterized SQL gives the database an enforced distinction between instructions and values. Marking text as untrusted inside a prompt does not create an equivalent boundary. The UK’s NCSC explains why controls also need to limit what a successful prompt injection can cause. NCSC’s analysis

Our SafeCall project validates model responses and checks known text patterns. It can catch malformed output and some unwanted content, but it does not enforce tool permissions. An unauthorized request can still be correctly formatted and pass a text filter.

Memory can carry the same problem into later work. A September preprint demonstrated attacks that caused agents to save malicious instructions from external content into persistent memory. Those instructions influenced later sessions and caused information leakage in controlled experiments. Persistent memory poisoning research

An instruction can also lose its origin when an agent summarizes it. Another agent receives the summary and may have no indication that part of it came from an untrusted document. Storage permissions and version history help control changes and investigate them. We also have to preserve the source and its trust level through summaries and handoffs. Knowing which agent saved something does not establish that the instruction deserves to be followed.

Code execution

Running code gives an agent access to whatever the execution environment exposes. A restricted environment can still contain valuable credentials, mounted files and connections to internal services.

gVisor adds isolation between applications and the host operating system, reducing exposure to host compromise. That protection does not remove the access deliberately provided inside the sandbox.

The October OpenHands advisory illustrates the difference. Versions of its Software Agent SDK before 1.52.0 could expose the sandbox agent server without authentication. An attacker able to reach the published port could execute commands as root inside the sandbox, access the mounted workspace and read stored secrets. Escaping to the host was not required. VulnCheck’s advisory

An unauthenticated service is a security failure we already understand. In an agent deployment, the review still has to cover the service controlling execution and everything available inside the environment. The sandbox can be working while the data placed inside it remains exposed.

Dependencies and external tools

The packages and services around an agent need the same scrutiny as other software. Trivy identifies known vulnerabilities in supported components. Our Security Watch alpha compares advisories and security news with a declared inventory and records the evidence behind its findings. Missing versions and unsupported matching cases remain uncertain.

A tool can also supply malicious instructions without a known vulnerability for a scanner to match. In August, Pillar Security observed an MCP server behaving normally for three tool calls, then changing subsequent tool and prompt metadata to include credential-seeking instructions. Researchers confirmed the behaviour and corresponding source code; the report did not establish successful credential theft from victims. Pillar’s Deadbugz research

The initial tool description would not have shown that later behaviour. A remote service can also change without a package update on our side, so we need to track relevant metadata changes and restrict the operations available to the tool. Outbound access needs similar attention. Permission to contact an email or storage service does not establish which recipient may receive confidential data.

Approvals, retries and stopping work

A human approving a plan may not be approving the action the agent eventually chooses. For sensitive operations, approval needs to cover the actual target and arguments and remain tied to that operation through execution.

Our Agent Guard supports scoped task grants, delegation checks and approval of the exact registered action. The embedded guard covered when this article was first published keeps its authorization state inside one process. Restarts lose its approval-consumption and usage history, while separate instances do not share those records. A service receiving delegated work still needs its own authorization check.

Update, 5 November 2026. Version 0.1.0-alpha.2 adds a separate executor that preserves authorization state in an encrypted local SQLite ledger. We tested its restart and recovery behavior in Agent Guard and the permission to try again. Independent executors still need coordinated authorization.

Retries can cause damage even when the original action was authorized. A remote service may complete a change before the caller times out. Repeating the request can perform the operation twice. The receiving system needs to recognize duplicates, and the caller needs to preserve uncertainty when it cannot establish what happened.

Stopping the parent agent does not necessarily stop delegated or queued work. Agent Guard’s cancellation is cooperative; it cannot forcibly stop arbitrary code. We need to know which work is still running and which credentials or grants remain usable after the parent stops.

How we check the coverage

The familiar names of these risks can hide what we need to test. A permission check, an inventory match and a tool response each use different evidence. We want to know whether the system notices when the evidence changes, even if the account name or the final message looks the same.

Our follow-up tests, added here on 5 November, give us three concrete checks.

• Keep the final worker, tool and deployment revision unchanged. Change the authority behind the request. An approved release and a read-only investigation need different decisions. So does a release whose intermediate handoff narrowed its scope to staging. The six cases in part two show why the reviewer needs that history.
• Keep the declared fixed package version unchanged. Make the inventory too old to support it. The finding should become uncertain. Our four-entry inventory test produced that difference, despite the identical version number.
• Keep the error response unchanged. Check whether the backend made a change before returning it. In our Agent Guard tests, one TOOL_FAILED followed a refusal to write, while another followed a completed write. Retrying both as though nothing happened would ignore the evidence that matters.

This is also how we would assess an AI reviewer. Give it enough history to distinguish those cases, then measure whether it does. The six-case exercise is a starting point, with explicit synthetic records. Measuring missed attacks or unnecessary interruptions in real work needs a separate evaluation. The permission checks still have to hold when the reviewer gets the judgment wrong.

Continue to part two: what we cover with Security Watch and Agent Guard.