
For over two decades, the API Gateway has been one of the most important pieces of modern application architecture. Whether you’re building a mobile application, a SaaS platform, or a collection of microservices, chances are every request from your users passes through an API Gateway before it reaches your backend.
But something has changed.
The clients making requests are no longer just browsers and mobile apps. Increasingly, they’re AI agents, and AI agents behave very differently from traditional software. As a result, we are beginning to see the emergence of a new architectural component called the Agent Gateway.
To understand why it is needed, it helps to trace how gateways have evolved.
The API Gateway: A Reverse Proxy with API-Specific Capabilities
When I think about gateways in software, the first concept that comes to mind is a proxy.
A proxy is an intermediary between two systems. A forward proxy acts on behalf of a client, while a reverse proxy sits in front of one or more servers and acts on their behalf.
At its core, an API Gateway is a reverse proxy with capabilities designed specifically for managing APIs. That may sound like an oversimplification, but it is a useful mental model.
An API Gateway sits between API consumers and the services that fulfill their requests.
Imagine a user opening a food-delivery application and tapping Order food. The application sends a request through an API Gateway, which routes it to the appropriate backend services.

The gateway may handle routing and load balancing, authentication and authorization, rate limiting and quotas, TLS termination, Protocol translation, Request and response transformation, Caching, Logging, tracing, and observability, API versioning and lifecycle policies and many more.
Most importantly, the API Gateway makes a fundamental assumption about its client:
The client already knows which API it wants to call.
A mobile app doesn’t ask “How do I order food?”. Its developers have already encoded that logic into the application. The client knows that it needs to send a request such as:
The API Gateway’s job is to authenticate, govern, route, and reliably deliver that request at scale.
AI Agents Don’t Work That Way
Now imagine replacing that mobile app with an AI agent. Instead of receiving a predetermined instruction to call a specific endpoint, the agent receives a goal. “Book me a hotel near the conference venue under $250.”
To accomplish this, the agent might need to:
- Identify the conference venue
- Search multiple hotel providers
- Compare prices and cancellation policies
- Read reviews
- Calculate travel time
- Check room availability
- Ask the user to clarify missing preferences
- Reserve the selected room
- Send a confirmation
The exact sequence is not necessarily known in advance. It may change depending on the information returned by each tool, but the agent must repeatedly decide:
- Which tools are available?
- Which tool is appropriate for this step?
- What arguments should I send?
- Do I have enough information to proceed?
- Should I retry a failed operation?
- Should I select a different provider?
- Is user approval required?
- Has the overall goal been completed?
This is fundamentally different from traditional applications. Instead of executing workflows that developers have already mapped out. An AI agent determines parts of the workflow dynamically at runtime. That flexibility is what makes agents powerful. It is also what makes them difficult to control.
The LLM Gateway: API Gateway Capabilities for Model Traffic
As large language models became widely adopted, organizations encountered a new set of production concerns. Applications were no longer calling only conventional APIs. They were also sending prompts to multiple model providers, receiving probabilistic outputs, consuming tokens, and incurring variable costs.
This led to the emergence of the LLM Gateway, sometimes called an AI Gateway.
An LLM Gateway applies familiar gateway patterns to model inference traffic. It can centralize capabilities such as:
- Model and provider routing
- Fallbacks and retries
- Token-based rate limits
- Prompt and response logging
- Cost and usage tracking
- Semantic caching
- Prompt guardrails
- PII and secret redaction
- Content filtering
- Load balancing across models
- Latency and quality monitoring
For example, an LLM Gateway might route simple classification requests to a smaller, less expensive model while sending complex reasoning tasks to a more capable one. It might switch providers when a model is unavailable, enforce a team’s monthly budget, or redact sensitive information before a prompt leaves the organization.
These are important production capabilities, but the primary object being managed is still the model request.

An LLM Gateway helps an application use models reliably, securely, and cost-effectively.
An agent, however, depends on much more than a model.
The MCP Gateway: Managing Access to Tools and Context
Anthropic introduced the Model Context Protocol, or MCP, in November 2024 as an open standard for connecting AI applications to external tools and data sources. Since then, MCP has become an important part of the agentic ecosystem.
Instead of creating a proprietary integration for every agent and every external system, developers can expose capabilities through MCP servers. An agent can then use those servers to search a database, retrieve a document, create an issue, send a message, query an API, or perform another action.

However, standardizing the protocol does not automatically solve the operational problems involved in managing a large tool ecosystem.
Different MCP servers may represent different security boundaries. They may depend on separate authorization servers, credentials, scopes, network policies, and downstream APIs. One MCP server may use an API key for authentication, while another uses OAuth 2.0, and another OAuth 2.1 + DCR. This makes centrally managing auth difficult. The MCP specification recommends OAuth-based mechanisms, but deployment models and implementation maturity can still vary across servers and a lot of MCP servers today are not fully complaint with the spec.
As organizations connect agents to tens or hundreds of tools, several challenges emerge:
- How are users and agents authenticated consistently?
- Which MCP servers should each agent access?
- Which tools within a server are permitted?
- Where are credentials stored and exchanged?
- How can tool calls be audited centrally?
- How do you revoke access across many servers?
- How do you prevent sensitive tool results from leaking into prompts?
- How do you avoid loading hundreds of irrelevant tool definitions into the model’s context?
The final problem is particularly important. Tool definitions and tool results consume context. As the number of available tools grows, blindly exposing all of them to the model can increase token consumption, latency, and the likelihood that the model selects the wrong tool. Tool discovery and selective loading therefore become operational concerns, not merely prompt-engineering concerns.
An MCP Gateway provides a centralized control point between MCP clients and MCP servers.

Depending on the implementation, it can provide:
- Centralized authentication and credential brokering
- Authorization and scope enforcement
- MCP server registration and discovery
- Tool filtering
- Tool namespacing
- Request and response inspection
- Audit logging
- Rate limiting
- Policy enforcement
- Tool-definition caching
- On-demand tool discovery
- Protection against malicious or untrusted tool output
The MCP Gateway governs access to MCP-based capabilities. But MCP servers are still only one part of an agentic system.
The Agent Gateway
An AI agent may depend on some or all of the following:
- One or more large language models
- APIs
- MCP servers and tools
- Databases and knowledge stores
- Short-term and long-term memory
- Other specialized agents
- Human approval workflows
- Identity and policy systems
An Agent Gateway is an intelligent middleman between an AI Agent, and all the components it relies on. Agent Gateways provide a centralized control plane and enforcement point across these interactions. These can include, LLMs, MCP Tools, Memory, other Agents, and of course, an API.

Like an API Gateway, an Agent Gateway may proxy requests, enforce access controls, apply policies, limit traffic, and collect telemetry.
Like an LLM Gateway, it may govern model selection, token usage, prompts, responses, safety controls, and cost.
Like an MCP Gateway, it may manage tool discovery, credentials, permissions, and tool invocation.

But its scope is broader than any one of those categories. It governs the agent’s interactions as part of a complete runtime workflow.
An Agent Gateway may be responsible for:
- Establishing and propagating agent identity
- Discovering and exposing relevant tools
- Enforcing policies before and after tool calls
- Routing requests across models, APIs, tools, and other agents
- Redacting sensitive data
- Requiring human approval for high-risk actions
- Applying budget, token, and execution limits
- Detecting loops and abnormal behavior
- Recording end-to-end traces
- Coordinating access to memory and context
- Evaluating whether an action is permitted in the current state
- Terminating an agent run when safety or cost thresholds are exceeded
- …and many more.

Why Agentic Systems need a Gateway
Agent identity is still an unsolved operational problem
Traditional API security is usually based on a relatively clear identity chain.
A user authenticates to an application. The application receives a token. The token identifies the user, the application, or both. Backend services validate that identity and enforce the corresponding permissions.
Agents complicate this model. An agent may be acting on behalf of a user, running as an independent workload, invoked by another agent, operating across multiple sessions, delegating work to subagents, using shared infrastructure, or even calling tools that require different identities.
When an agent invokes a tool, which identity should the tool evaluate? Is it the identity of the user, the application, the agent, or the organization? What happens when one agent delegates a task to another? Which permissions should be transferred, and for how long?
Without a consistent identity and delegation model, systems tend to fall back to shared API keys or overly broad service accounts. That makes least-privilege authorization difficult and weakens accountability.
An Agent Gateway can provide a consistent point for establishing agent identity, propagating user context, exchanging credentials, narrowing scopes, and recording who or what initiated each action.
Agents expand the security boundary
A traditional chatbot primarily generates text. An agent can generate text and then use that text to take action. It may send an email, modify source code, issue a refund, retrieve customer records, deploy an application, or initiate a payment.
This creates risks that do not exist in an ordinary API traffic:
- Prompt injection can influence tool selection.
- Untrusted tool output can manipulate subsequent reasoning.
- A model can generate valid but unsafe tool arguments.
- An agent can combine individually harmless operations into a harmful sequence.
- Credentials may cross boundaries between users, agents, tools, and models.
- A compromised tool can return instructions disguised as trusted data.
Authentication alone does not solve these problems. A request can be properly authenticated and still be unsafe. Agentic systems need policies that evaluate more than the caller and endpoint. They may need to consider the user’s intent, the selected tool, the arguments, the current workflow state, the sensitivity of the resource, and the potential consequence of the action.
An Agent Gateway can enforce controls at these boundaries. For example, it might allow an agent to search financial transactions but require human approval before issuing a refund.
Agents introduce stateful workflows
An individual LLM request is generally stateless i.e the model receives an input and produces an output.
An agentic workflow is stateful and may maintain conversation history, a task plan, previous tool results, user preferences, intermediate artifacts, approval status, retry counts, budget consumed, and long-term memories.
This does not mean the gateway itself must store all agent memory. In many architectures, memory will remain in dedicated databases, vector stores, or state-management services. The gateway’s role is to govern access to that state.
It can determine which agent may read or write a memory, what context may be sent to a model, which information must be redacted, and whether state from one user or session can be reused in another. This distinction matters. An Agent Gateway does not need to become the memory database.
It needs to become the policy and visibility layer through which memory is accessed.
Agentic observability is difficult
API observability typically focuses on individual requests, such as which endpoint was called, how long it took, what status code was returned, and which service handled the request.
LLM observability adds another set of questions, including which model was used, how many tokens were consumed, what the request cost, and whether a guardrail was triggered. Agentic observability must connect all of these events into a coherent execution trace.
For a single user request, an agent might make several model calls, invoke multiple tools, query memory, delegate to another agent, retry failed operations, and wait for human approval.
When the final result is wrong, it is not enough to know that one API returned a 200 OK; you need to understand the goal the agent received, the plan it formed, the tools available, why it selected a particular tool, which arguments it generated, what the tool returned, how that result affected the next decision, where time and tokens were spent, which policy decisions were applied, which user, agent, and credential were involved, and where the workflow failed.
An Agent Gateway can capture these interactions at a shared boundary and correlate them into an end-to-end trace.
Autonomous execution needs limits
A traditional application usually has bounded execution paths, but an agent can keep planning, calling tools, evaluating results, and retrying until it reaches a stopping condition. Without clear limits, this can lead to long or infinite loops, repeated tool calls, runaway token usage, escalating costs, duplicate side effects, cascading agent-to-agent delegation, and repeated failed authentication attempts.
Conventional request-per-second limits are not enough for agentic systems. They need controls that account for the full workflow, including token and cost budgets, tool-call volume, delegation depth, execution time, retries, concurrency, and the sensitivity of the requested operation.
The Agent Gateway can enforce these limits across the entire execution, rather than evaluating each request in isolation.
From Managing Requests to Governing Intent
API Gateways were designed for a world in which applications knew which endpoints to call.
LLM Gateways emerged when model inference introduced new routing, cost, privacy, and safety requirements.
MCP Gateways emerged as agents gained access to a growing ecosystem of tools and context providers.
Agent Gateways are the next step in that evolution.
They are needed because the unit being governed is no longer only an API request, a model invocation, or a tool call. It is an agentic execution, a sequence of decisions and actions taken dynamically to achieve a goal. That shift alone changes the role of the gateway.
The gateway is no longer concerned only with:
Is this client allowed to call this endpoint?
It must also help answer:
Is this agent allowed to take this action, with this tool, using this identity, in this context, at this point in the workflow?
APIs are not disappearing. They remain the foundation on which agents act, but the clients consuming them are changing. They can reason, choose tools, delegate work, maintain state, and take consequential actions.
As clients evolve, the gateway must evolve with them.
Building the Future: Fabric Gateway
At Postman, we are building what we believe is the next generation of this architecture, we call it the Fabric Gateway.
Fabric Gateway is designed to bring together the capabilities discussed in this article – API governance, model routing, MCP tool management, and agent-level policy enforcement, into a unified control plane for agentic systems.
Our goal is simple: make it possible to safely run agents in production without sacrificing flexibility, speed, or developer experience.
We are currently opening up early access to teams that are building agentic applications and want to help shape this new category.
If you are exploring agents in production, or thinking about how to govern them at scale, we would love to hear from you.
Join the early access and help define what the Agent Gateway should become.
The post Agent Gateway: The Next Evolution of the API Gateway appeared first on Postman Blog.