withHumanNov 10, 202612 min read

What is an MCP gateway? How it works and when you need one

An MCP gateway sits between AI agents and the MCP servers they call. It holds credentials, filters tools, applies policy to each call and logs it. Here's how it works.

Connect three agents to five MCP servers and you have fifteen connections. Each one has its own credentials, usually an API key in a config file on someone's laptop, and nobody can say which agent called which tool last Tuesday.

An MCP gateway replaces those fifteen connections with one. This page explains what an MCP gateway does, how it handles a tool call, where it stops protecting you, and when you actually need one.

What is an MCP gateway?

An MCP gateway is a server that sits between AI agents (MCP clients) and the MCP servers they use. Agents connect to the gateway instead of to each server. The gateway forwards their tool calls to the right server and applies the organization's rules on the way: who can use which tools, with which credentials, and which calls need approval.

To the agent, the gateway looks like one MCP server with a lot of tools. To each MCP server, the gateway looks like one client.

The MCP specification doesn't use the word "gateway". Its closest term is an MCP proxy server, an MCP server that connects clients to other APIs. In practice, vendors use "proxy" for something that fronts a single server and "gateway" for something that fronts many servers and adds policy. The line is blurry, and you'll see both words for the same product.

How an MCP gateway works

Every MCP session goes through the same two steps: the agent asks for the tool list, then calls tools from it. A gateway intercepts both.

Listing tools. When the agent sends tools/list, the gateway asks each server it fronts for its tools, drops the ones this agent isn't allowed to use, and merges the rest into one list. Tool names get a server prefix so two servers can both have a search tool without colliding. GitHub's create_pull_request becomes github__create_pull_request.

Calling a tool. When the agent sends tools/call, the gateway reads the prefix to find the server, checks the call against its rules, attaches that server's credential and forwards it:

{
  "jsonrpc": "2.0",
  "id": 7,
  "method": "tools/call",
  "params": {
    "name": "stripe__create_refund",
    "arguments": { "payment_intent": "pi_3QxY2L", "amount": 4900 }
  }
}

The gateway sees the exact tool and arguments before anything reaches Stripe. This is why the gateway sits in the path. It can refuse the call, hold it for a person, or let it through and log the result.

MCP also has resources, prompts, sampling and elicitation. Gateways vary a lot in how much of that they forward. Many handle tools only, because tools are where the actions happen.

What an MCP gateway does

Holds the credentials

This is the job that matters most, and the one people underrate.

Without a gateway, every agent holds the credentials for every server it uses. A personal access token sits in ~/.cursor/mcp.json or an environment variable, readable by the agent and by anything else running as that user.

With a gateway, the agent holds one credential: its own, issued by the gateway. The gateway stores the GitHub token, the Stripe key and the Linear OAuth grant, and attaches the right one to each forwarded call.

The MCP spec effectively requires this split. It forbids "token passthrough", where a server accepts a token meant for another service and forwards it downstream: "MCP servers MUST NOT accept any tokens that were not explicitly issued for the MCP server." A gateway that forwards the agent's GitHub token to GitHub is doing exactly that.

Gateways usually support two kinds of downstream credentials:

Shared credentials belong to the organization. A support bot calling Stripe with the company's restricted key is the typical case.
Per-user credentials belong to the person the agent works for. Your coding agent opens pull requests as you, not as a shared bot account, so GitHub's own permissions still apply.

Controls which tools each agent sees

The gateway decides which tools appear in each agent's tool list. A support agent sees stripe__get_payment and stripe__create_refund. A coding agent sees GitHub and Linear and nothing from Stripe.

Hiding a tool is stronger than telling the model not to use it. A tool the agent can't list, it can't call. It also keeps tool lists short. A long list of similar tools makes it harder for the model to pick the right one.

Applies policy to each call

Tool filtering is all-or-nothing per tool. Policy works per call, on the arguments: allow refunds under $50, deny any GitHub call that targets main, send everything that touches production to a person for approval.

This is where gateways differ the most. Some only allow or deny. Some can hold a call until a human approves it, which is the human-in-the-loop pattern applied at the tool-call layer.

Logs every call

Because every call passes through one place, the gateway can record which agent called which tool, with which arguments, when, and what happened. Without a gateway, that record is scattered across each agent's local logs, if it exists at all.

Runs and routes servers

Some gateways also run the servers. They start local MCP servers in containers, restart them when they crash, and route each session to the server instance that holds its state. This is mostly an infrastructure concern, and it's what separates a Kubernetes-oriented gateway from a policy-oriented one.

The catch: a gateway only governs calls that go through it

An MCP gateway controls the MCP route to a service. It doesn't control the other routes.

If a coding agent can read GITHUB_TOKEN from its environment, it doesn't need the gateway's github__merge_pull_request tool. It can run gh pr merge or curl the GitHub API directly, and the gateway never sees the call. The same goes for a Stripe key in a .env file the agent can open.

So the gateway is only as strong as the credential cleanup that comes with it. When you move a service behind a gateway:

Take that service's credentials out of the agent's environment: env vars, config files, CLI logins.
Turn off any direct integration the agent runtime has for the same service. Some agents ship with their own built-in connectors.
Give the gateway credentials scoped to what agents should do. If the token can delete repositories, the gateway's policy is the only thing stopping it.

This is the step that's easiest to skip, and skipping it leaves a gateway that agents can walk around.

MCP gateway vs. API gateway vs. AI gateway

These three get mixed up, partly because some products are all three.

	MCP gateway	API gateway	AI gateway (LLM gateway)
Sits between	Agents and MCP servers	Clients and HTTP APIs	Applications and model providers
Understands	MCP sessions, tool lists, tool calls and arguments	HTTP routes, methods and headers	Prompts, completions and tokens
Typical policy	Which agent can call which tool, with which arguments	Rate limits, authentication, routing	Model choice, token budgets, fallbacks, prompt filtering
Example question it answers	"Can this agent refund $4,900?"	"Is this client allowed to call `POST /refunds`?"	"Which model should handle this prompt, and has the team hit its budget?"

You can put an API gateway in front of an MCP server that uses the Streamable HTTP transport. It will authenticate and rate-limit the traffic, but it sees JSON-RPC bodies rather than tools. It can't filter a tool list or make a decision based on a tool's arguments without MCP-specific logic.

An AI gateway sits on the other side of the agent, between the agent and the model. It never sees tool calls being executed, only the model asking for them.

Types of MCP gateways

Gateways fall into four groups, depending on the problem their builders started from.

Local gateways run MCP servers in isolated containers on a developer's machine. Docker MCP Gateway is open source under the MIT license, runs catalog servers in containers and keeps secrets out of their environment variables.
Infrastructure gateways put MCP traffic behind the identity, network and policy layer a team already runs. Microsoft's MCP Gateway is a reverse proxy for Kubernetes with session-aware routing, so every request in an MCP session reaches the same server instance. Kong and the cloud providers' API gateways add MCP support to the gateway you already run, and Amazon Bedrock AgentCore Gateway also turns existing APIs and Lambda functions into MCP tools. These gateways decide who can call which tool, then allow or block the call.
MCP platforms add a catalog of approved servers, server hosting and guardrails on top of the gateway.
Approval gateways decide each call against rules, and can hold it until a person approves it. withHuman's MCP Gateway is in this group. It keeps each server's credentials out of the agent's environment and sends every call through an approval pipeline.

These groups overlap and can stack, and the right one depends on whether your problem is running servers or controlling what agents do with them. For a product-by-product comparison, see the best MCP gateways for AI agents.

When do you need an MCP gateway?

You probably don't need one if you're a single developer with a couple of MCP servers on your own machine, and the tools are read-only or easy to undo.

You probably need one when any of these is true:

Several agents or people share the same servers. Managing credentials and tool lists per laptop stops working around the second team.
Agents hold credentials that can do real damage. A key that can issue refunds, merge to main or write to a production database shouldn't live in an agent's environment.
You need to answer "who did this?" Security reviews, incident response and regulation such as the EU AI Act's human oversight requirements all need a record of what agents did and who approved it.
Some calls should wait for a person. Reads can run freely, but refunds, deploys and deletes need approval.

Human approval at the MCP gateway

withHuman's MCP Gateway is built for that last case. The gateway is a natural place for approval: it sees the exact tool and arguments before the call leaves, and it holds the credential, so a call it doesn't forward can't run.

Here's how the parts above map to it:

Credentials. Each server's credentials stay in withHuman. Agents connect with their own withHuman credential, and the withhuman CLI sets that up for Claude Code, Codex and other runtimes. A server can use the organization's credential or each member's own account.
Tool access. Each agent sees only the tools you grant it, named <server-slug>__<tool>.
Policy and approval. Every call the agent is allowed to make goes through your approval pipeline. The pipeline approves it, denies it, or sends it to a person through an escalation path. A denied or expired call never reaches the server.
Audit. Every call is an approval request, so the request list and the audit log record its decision, who made it and which agent instance asked.

One design choice is worth explaining. Teams often want an agent to read from a service freely but have every write approved. Classifying each tool as a read or a write is error-prone, so withHuman's MCP Gateway doesn't do it. Instead, a server can have several scopes, each a separate credential, such as a read-only token and a write token. The agent picks one on every call in a withhuman_scope argument, and your pipeline decides on that:

MCP server	`withhuman_scope`	Pipeline result
`cloudflare`	`org_read`	Approve automatically
`cloudflare`	`org_write`	Send to the head of infrastructure
`cloudflare`	`personal_write`	Send for review

The service enforces the difference. A write tool called with the read token fails at Cloudflare, whatever the agent intended. A read tool called with the write token waits for a person. Agents can also add a short withhuman_context explanation to any call. Reviewers see it next to the arguments, and pipeline rules never read it, so an agent can explain a call but can't talk its way past a rule.

The limits, so you can tell whether it fits:

The MCP Gateway is in the hosted edition only. The open source edition gates calls through hooks in each agent instead.
It forwards tools only. MCP resources, prompts, sampling and elicitation aren't supported.
Servers must be remote and speak Streamable HTTP. To use a server you run yourself, connect it through a gateway tunnel.
It can't stop an agent from using credentials it still holds. Move them out of the agent's environment as described above.

To try it, follow Set up MCP Gateway. It walks through connecting Linear to Codex, and your coding agent can do most of the setup for you.

FAQ

What is the difference between an MCP server and an MCP gateway?

An MCP server exposes one set of tools, such as GitHub's or Stripe's. An MCP gateway sits in front of many MCP servers and gives agents one endpoint for all of them. It adds credential storage, tool access control, per-call policy and logging.

Is an MCP gateway the same as an MCP proxy?

Mostly, yes. The MCP specification uses "MCP proxy server" for an MCP server that connects clients to other APIs. "Gateway" usually means a proxy that fronts many servers and adds access control and policy. Products use both names.

Are there open source MCP gateways?

Yes. Docker MCP Gateway is open source under the MIT license, and Microsoft's MCP Gateway is on GitHub. Both focus on running and routing MCP servers. agentgateway, Obot and IBM ContextForge are open source too. withHuman is open source under Apache 2.0, but its MCP Gateway is part of the hosted edition. Our MCP gateway comparison covers what each one is best at.

Does an MCP gateway protect against prompt injection?

Not directly. A gateway doesn't stop a model from reading malicious instructions in a web page or an issue. It limits what an injected agent can do afterwards: tools it can't list, it can't call, and consequential calls can wait for a person who sees the exact arguments.

Can an agent bypass an MCP gateway?

Yes, if it holds another way to reach the same service, such as an API key in an environment variable or a logged-in CLI. Remove those credentials from the agent's environment so the gateway is the only route.