Introducing withHuman

Today we're launching withHuman, an approval layer for AI agents. It sits between your agents and the tools they use. You write rules that approve or deny calls automatically, or send the risky ones to humans.

We put out a short manifesto a little while ago. This is the longer version: how we got here, what we've built so far, and what we're still figuring out.

Why we built this

Before withHuman, we built Metoro, an AI SRE agent. When something breaks in production, Metoro goes and figures out why, then works out a fix. The fixes are mostly the unglamorous things an on-call engineer does at 3am: roll back the deploy, restart the thing that's crash-looping, scale something up, change a config value.

Metoro turned out to be pretty good at working out the fix. Getting customers to let it actually apply the fix was a different story. And honestly, we understood why. Rolling back a deploy is usually fine. Usually. Nobody wanted to hand an agent admin on their production cluster and find out which kind of rollback this one was going to be.

So we did what most agent companies end up doing and built an approval step. Metoro would propose an action, the action would sit in the Metoro UI until someone approved it, and then it would run. That helped a lot, and customers started letting it do more.

But three things kept coming up.

1. We kept getting asked to encode other people's org charts. Who's on call for this service? Is a restart fine during the day but not at night? Does the payments team want to see every change to their namespace? Good questions, but they weren't really about Metoro. They were about how each customer runs their company, and every customer answered them differently.

2. The permissions were too blunt. You could let the agent restart pods, or not. There wasn't a good way to say "restarts are fine, unless it's the payments namespace, in which case ask someone on payments". So most teams picked the safe option, and the agent spent a lot of its time making suggestions rather than actually fixing things.

3. Metoro was never the only agent. Our customers also had coding agents on every laptop, a support agent in the helpdesk, and a few internal agents someone had built over a weekend. Each one had its own idea of approvals: permission prompts in Claude Code, tool approvals in an agent SDK, a Slack bot somebody wrote, or nothing at all. So a team with five agents had five separate sets of approval settings, all working slightly differently. If you wanted to know what your agents were allowed to do, you had to go and check each one.

We'd built a decent approval flow for one agent. What people actually needed was one for all of them.

What we think the answer looks like

We don't think anyone has this figured out yet, us included. But a few ideas have held up through Metoro and the last few months of building withHuman, so they're what we've built around.

First, approval should happen at the tool call. The moment that matters is when the agent actually goes to issue the refund or run the migration, so that's where we stop it. The approval is transparent to the agent. It calls its tool like it normally would, something wrapping the tool holds the call while a decision gets made, and then the call either runs or fails. Nothing about how the agent works has to change. And because the agent isn't part of the approval, it can't forget to ask, or be talked out of asking by a cleverly worded prompt.

Second, most calls shouldn't need a person. If every refund pings someone, people quickly start approving without reading, and then you've got the delay of human review without the actual review. We saw this with coding agents. They asked before almost everything, and plenty of people ended up aliasing claude to claude --dangerously-skip-permissions. So the default should be automatic. Rules let the routine stuff through, refuse the obviously wrong stuff, and only send people what's left. The $10 refund just happens and the $10,000 one waits. Some rules are simple checks on the tool and its arguments. Others need a bit of judgment, and for those it seems reasonable to ask a model, or a service of your own, before bothering a human.

Third, the rules should belong to the organization, not to each agent. Support should decide how refunds get approved, regardless of which agent is issuing them. In the same way, sales should decide what agents can change in the CRM, and engineering should decide what they can do in production. You shouldn't have to rebuild all of that every time someone adopts a new agent or framework.

Fourth, the human part has to work. A request nobody sees is just a slow timeout. It should go to whoever's actually responsible, somewhere they'll notice it, with enough context to decide in a few seconds. If they don't answer, it should go to someone who will.

Last, the approval mechanism should be an open protocol. If every agent needs a custom integration with every approval system, we're back to five permission systems.

What we're starting with

There are two main pieces.

The first is the Agent Approval Protocol (AAP), a small open spec which defines how agents should ask for a decision and wait for the answer. We kept it separate from withHuman because we'd genuinely like other people to implement it, adopt it and contribute.

It comes with ready-made ways to use it: prebuilt hooks for coding agents like Claude Code and Codex, SDKs for Python and TypeScript, and higher-level libraries with drop-in integrations for most popular agent frameworks, like Deep Agents, LangGraph, Pydantic AI and Mastra.

The second is withHuman itself. It's the centralized place where you write your rules and set who gets asked when a call needs a person. It's open source, and there's a hosted version that we run.

The overall flow looks like this:

Your agent makes its tool calls as normal. Before a call reaches the tool, the AAP layer (a hook or one of the SDKs) holds it and sends withHuman a request describing the call.

withHuman runs the request through your approval pipelines. These are your rules. They can approve the call, deny it, ask a model or a service of your own, or decide it needs a person. If it does, an escalation path picks who to ask, like whoever's on call for the team that owns it, and notifies them wherever suits them.

Once there's a decision, withHuman sends it back. An approved call runs as normal. A denied one fails, and the agent sees that it was denied and the reason why.

If you'd like to give it a go, the Quickstart connects a coding agent you already use in a few minutes.

What we're still figuring out

Quite a lot, honestly, and we keep coming back to a few things.

The first is tool calls beyond MCP. We're starting with MCP tool calls, because it's a well-used standard and an easy place to hook into. But plenty more needs gating too, like HTTP requests an agent makes through a CLI or a script.

Then there's context for reviewers. A tool call on its own often isn't enough for a reviewer to decide well. Its arguments can be full of opaque identifiers, like a customer ID or a commit hash, that mean nothing until someone looks them up. Ideally they'd be turned into something meaningful, like the customer's name and plan, before the reviewer sees the request.

Finally, there are the credentials agents can reach. An agent that holds its own credentials can always go around an approval layer. If a coding agent can read your GITHUB_TOKEN, it can skip the gated tool and call the GitHub API directly with curl. Robust approvals require removing credentials from the agent's environment. For MCP servers, the built-in MCP Gateway does this: it holds the credentials, so agents never see them. Other credentials, like CLI tokens or API keys in environment variables, are harder.

If you've got opinions on any of this, we'd love to hear them. Open an issue on GitHub, or just try it with your own agents and tell us what broke.

We started all this because we wanted to let our own agent do more. We hope it lets yours do more too.