AI agents are software with hands: they read your documents, call your APIs, and act in your systems. Bullet Proof Software assesses and hardens that new attack surface – the SoftwareMile specialist practice for secure engineering, applied to AI.
Threat Modeling for AI Agents
What can the agent read? What can it change? Who can make it do so? We map agent capabilities, tool access, token scopes, identities, and data flows – then work through misuse cases: prompt injection, data exfiltration through tool output, privilege escalation via chained tools, and poisoned context sources.
MCP Server Security Review
MCP servers are security boundaries between AI and your infrastructure. We review authentication and transport, tool scoping and input validation, secrets handling, logging sufficiency, and whether approval gates actually gate the consequential actions.
Guardrail and Permission Design
Least-privilege tool design, human-approval workflows for irreversible actions, audit trails that reconstruct what an agent did and why, and evaluation criteria that catch unsafe behavior before rollout.
Independent of the Builder
We review agent systems whether or not our sister practice SoftwareDepo built them – and when it did, this review is independent by design: different practice, different engineers, different incentives.
Describe your agent or MCP deployment – platforms, tools exposed, data sources, and what the agent is allowed to do. We will scope a review with clear deliverables.
Related services
- Penetration Testing and Security Assessment — Adversarial testing of agent and MCP deployments: penetration testing and security assessment.
The Three Properties That Make an Agent Exploitable
Three properties together create the problem. The agent has access to something sensitive. It processes content an outsider can influence. It has some route to send information outward. With all three present, an attacker who can plant text in a shared document, a web page, a code comment, a support ticket or an email can direct the agent to retrieve data and hand it over, without ever touching your infrastructure directly. Two of the three is not a safe configuration either. Data still leaves through channels no architecture diagram shows, including error text, retry behavior, and whatever the agent renders back to a user who is not the person the data belongs to. Mapping the three properties is where a review starts, because removing one of them is a smaller engineering problem than defending the combination.
Failure Modes the MCP Specification Does Not Currently Prevent
Tool descriptions are model input. A server you connect supplies text that enters the model’s context and can be written to influence behavior, which makes connecting a server a trust decision about its maintainer and not only about its transport. A server can also change its tool definitions after you approved them, so what you reviewed and what runs next month may differ. Beyond that, the recurring issues are ordinary engineering problems.
- Tokens scoped far more broadly than the tool actually needs
- One shared service credential used for every user, so the audit trail cannot say who asked
- Credentials sitting in configuration files that get copied between machines
- Transports exposed further across the network than anyone intended
- Tool names colliding across servers, so the model selects the wrong one
- Approval prompts that show a tool name while hiding the arguments being passed
Approval Gates Fail Quietly
Human approval is the control most deployments lean on hardest, and it degrades with use. When every action requires a click, people click. The gates that survive contact with real work are reserved for consequential and irreversible actions, display the actual parameter values instead of a tool name, and default to deny when they time out. Reversibility is a useful sorting principle: an action you can undo cheaply rarely needs a person in the loop, while a payment, a deletion, an outbound message or a permission change does.
What a Review Covers, and Under What Terms
A review starts with an inventory: which tools the agent can call, what each tool’s credential can actually do, which content sources the agent reads, and where its output can go. From there it is permission modeling and targeted testing, because there is nothing to scan in the usual sense. Anything adversarial runs against systems named in the scope, under written authorization from a party entitled to grant it, inside a window agreed with your operations team, with a stop condition and a named contact on your side. Where the agent is connected to live business systems, that is a decision made deliberately or not at all. The terms are the same ones set out on our penetration testing and security assessment page.
Testing Something That Does Not Repeat
Security testing of agents is uncomfortable because identical input can produce different behavior. The input space cannot be enumerated, and no volume of testing establishes that an agent is safe. What is achievable is a maintained suite of adversarial cases run against every model, prompt and tool change: injection strings placed in each content source the agent reads, attempts to escalate through chained tools, requests that should be refused, and checks that permissions are enforced at the tool boundary instead of only described in the prompt. Treat the results as evidence about specific behaviors, and expect a model upgrade to invalidate some of it.
Log What the Agent Saw
Agent logging commonly records what the agent did. Reconstructing an incident needs what it read: retrieved documents and their sources, tool call parameters and results, the model and version in use, the system prompt in effect, and the identity the action was taken on behalf of. Without the retrieved context you can see a strange action and never learn what triggered it. This has privacy consequences worth deciding deliberately, since those logs will contain the same sensitive material the agent handles and need retention and access controls to match.
The Point Where This Stops Being Optional
Review earns its place when the agent gains write access, handles regulated or customer data, reads content originating outside your organization, or becomes available to people who are not on your team. Any one of those is enough on its own. Before that, a prototype running on synthetic data with read-only tools and a single internal user has very little to assess, and the work that pays off is writing down what the agent will be permitted to touch once it is real, so the permission model is not retrofitted around a design that has already shipped.
How MCP Connects, and What Each Transport Exposes
MCP defines two transports, and the security question is different for each. Over stdio the server runs as a child process on the same machine, launched by the client, exchanging JSON-RPC messages on stdin and stdout. There is no listening socket and no authentication step, because the trust decision was made when the process was spawned: the server runs under the same OS user as the client, with that user's file system and network access. Its credentials arrive as environment variables the client sets at launch, and implementations differ in whether the child receives a curated subset of variables or the parent's full environment, which decides whether one server can read another server's secrets.
Remote servers use Streamable HTTP, introduced in the 2025-03-26 revision of the specification to replace the earlier two-endpoint HTTP plus SSE transport. A single endpoint accepts POST for client messages and can upgrade to a server-sent event stream for server-initiated ones, with the session carried in an Mcp-Session-Id header. Authorization is layered on OAuth 2.1: the server behaves as a protected resource, answers an unauthenticated request with 401 and a WWW-Authenticate header pointing at its resource metadata document per RFC 9728, and the client runs an authorization code flow with PKCE against the authorization server it discovers there. Reviewing a transport comes down to a short list of mechanical questions.
- Whether a local HTTP server binds 127.0.0.1 rather than 0.0.0.0.
- Whether the Origin header is checked on every request, including the event stream, since a page in the user's browser can reach a loopback port by resolving an attacker-controlled hostname to 127.0.0.1.
- Whether the older HTTP plus SSE endpoints are still mounted alongside Streamable HTTP for backward compatibility, and whether both paths carry the same checks.
- Whether the server validates the audience of the access token it receives, instead of accepting any token its authorization server happened to issue.
- Whether a token presented to the server is being forwarded upstream to a third-party API rather than exchanged for one issued to that API.
What Tool Arguments Reach on the Other Side
A tool handler receives arguments the model produced, which from the handler's point of view is untrusted input arriving over an unusual channel. A tool that fetches a caller-supplied URL is a server-side request forgery primitive with a language model driving it: the cloud metadata endpoint at 169.254.169.254, an internal admin interface with no authentication because it was only ever reachable from inside, another service listening on loopback on the same host. A handler that builds a shell string out of an argument gives the model command execution. A file tool that joins a caller-supplied path onto a root directory without canonicalizing gives it path traversal, through ../ segments and through symlinks pointing outside the root. A tool whose argument is a SQL statement is an injection surface by design, and parameterization has nothing to bind when the whole statement is the parameter.
The fixes are the ordinary ones, applied at the point the argument enters. Resolve the hostname and test the resulting address against link-local and private ranges rather than pattern-matching the string, then repeat the check after every redirect. Pass an argument array to exec instead of handing a string to a shell. Canonicalize the path and confirm it is still under the root after resolution, not before. Give a query tool a database role with grants on named views, a statement timeout and a row cap, so the worst case stays bounded even when the statement is hostile. One thing here is not ordinary: a handler's error output becomes model context. Returning the raw exception hands the model a stack trace, an internal hostname or a connection string, and the model will carry that detail into later tool calls, which is how a probe that failed still tells an attacker what sits behind the tool.
Acting as the User Instead of as the Server
OAuth 2.0 Token Exchange, RFC 8693, is the mechanism that lets a server act on a user's behalf without holding a credential more powerful than the user. The server presents the user's token to the authorization server and receives a downstream token carrying the user's identity, with an act claim naming the server as the actor that requested it. Using it requires three things to line up: an identity provider that supports the urn:ietf:params:oauth:grant-type:token-exchange grant, a downstream API that accepts tokens from that issuer, and a downstream authorization model with per-user identities to map onto. Where a downstream system authenticates only with a static API key, delegation is not available at all, and the fallback is one narrowly granted credential per tool plus an authorization check inside the server before the call is made.
Retrieval breaks this even in deployments that get token exchange right. When documents are embedded into a vector index, the source system's access control list does not travel with the chunk, so a similarity search returns whatever sits closest to the query regardless of who asked. Two designs work. Query-time filtering stores the principals allowed to see each chunk as metadata and applies the filter using the requester's group membership. Per-principal indexes are easier to reason about and more expensive to maintain. Either way the permissions have to be re-synced when they change at the source: a document whose access was tightened last week is still retrievable from an index built before the change, and deleting a file from the source does not delete its embedding.
How the Server Got Onto the Machine
A local MCP server is usually one line in a client configuration file: a command, its arguments, and a set of environment variables. Often that command is npx -y some-package or uvx some-package, which resolves and downloads a package at launch. The consequence is that the code running today is not necessarily the code that was reviewed, and a compromised publisher account changes what executes on the workstation with no local edit and no prompt, since -y exists to suppress the prompt. Pinning an exact version, installing from a private registry mirror, or vendoring the server into a repository you control each turn that into a change visible in a diff.
The configuration file itself deserves treatment as a privileged artifact. It names commands that will run under the user's account whenever the client starts, so write access to that file is equivalent to code execution on the machine, and it is a natural target for anything that already has a foothold. Two practical questions follow: whether the file is managed centrally or edited by each engineer, and whether anything watches it for change. Running each server in a container with only the variables it needs addresses the related problem, which is that a server process on the host can read whatever the host user can read, regardless of what its tool definitions say it is for.
Frequently Asked Questions
Is this a penetration test or a code review?
It draws on both and is scoped as neither. It is not a model alignment evaluation: the question is not whether the model can be made to say something objectionable, but what your deployment lets it reach and where it can send what it reaches. It is not a compliance certification either, so the output is findings rather than an attestation. Where you own the MCP server source, that server is read as an application, and its tool handlers get the same treatment any request handler would.
Does this apply to MCP servers we did not build?
Yes, and the work shifts. For a server someone else operates, the review covers what is observable: the tool definitions it advertises, the arguments it accepts, the transport and authorization it requires, and the credential you hand it. What you cannot do is fix it, so the controls become containment. That means a dedicated credential per server with the narrowest downstream grant that still works, and a deliberate decision about whether that server's tool descriptions belong in the context of an agent that also has access to sensitive data.
What has to already exist on our side for the recommendations to be implementable?
An identity provider that can issue short-lived, audience-restricted tokens, and downstream systems that will accept them. Where a downstream system authenticates only with a static API key, per-user delegation is off the table and the design falls back to narrow per-tool credentials with checks in the server. Retrieval recommendations have a similar prerequisite: the content sources need permissions that exist as data somewhere, because an index cannot filter on an access rule that exists only as a folder naming convention.
Does a review need access to our source code?
No, but it changes the class of finding. Against a running deployment you can establish behavior: that a tool accepts a URL it should not, that a gate can be bypassed, that a document reachable by one user is retrievable by another. Source access establishes the reason, which is what a fix depends on: whether the address check runs before or after redirect resolution, whether the path is canonicalized before or after the join, whether the permission check sits in the handler or only in the client. For a hosted third-party server there is no source to read, and findings there stay bounded to what the interface exposes.
Can testing run against production?
Some behaviors exist only in production, because the real retrieval corpus, the real tool inventory and the real token scopes are what create the interesting paths, and a staging clone loaded with synthetic documents does not reproduce them. Where a finding can be established without a write, the test runs with read-only credentials. Where it cannot, the alternative is a sacrificial tenant or an account whose data is expendable. Exfiltration paths are best demonstrated with canary documents: uniquely identifiable strings planted in each content source the agent reads, so a successful path shows up as the canary arriving somewhere it should not and no real data moves.
What do you need from us before starting?
Mostly the grants as the provider's console actually reports them, rather than what they were intended to hold. Export the real policy documents: the IAM policy attached to each role, the OAuth scopes on each token, the database grants for each service user, because intended grants and actual grants routinely differ. A read-only account on the systems in scope saves a great deal of screen sharing.
Who on our side has to be involved?
The people who built the agent, plus whoever administers the identity provider or the cloud account. Narrowing a token's scope is a change in the identity provider or the cloud account rather than in the agent, so those findings resolve outside the agent codebase and need someone who can make them there. Retrieval findings resolve in the source system, which means the owner of the document store has to be reachable too. Replacing a shared service credential with per-user identity touches the identity provider, the server and every downstream grant at once, which makes it a structural change rather than a configuration fix, and it gets sequenced accordingly.