Inspect LLM responses and approve tool calls

This guide covers the response-inspection pipeline and the human-in-the-loop approval loop it feeds. Read it after Author and pull policy, which explains the scoping model (platform, org, group, agent) these keys reuse.

What is inspected, and where

The pipeline runs inside the enclave’s llmproxy, which already terminates every agent inference connection. It inspects responses: the model’s output that flows back to the agent, in both JSON bodies and streamed SSE, for the Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses wire formats. It detects tool calls the model wants to make (read-only detection over the decoded content, never the stop reason).

Where inspection is active, llmproxy buffers the whole response before the agent sees a byte (bounded by the proxy’s max-body cap; an over-limit response fails closed with an error, never a truncated forward). Where it is not active, nothing changes: an agent with no inspection policy streams byte-for-byte as before, with no buffering and no audit noise.

Detection and judging stay inside the enclave. Tool-call arguments never leave the enclave, and the controlplane never sees them. An operator working the approval loop sees the tool name and arguments; an auditor sees verdicts and tool names, but never arguments, in the audit events.

The verdict model

Detectors find tool calls and emit findings; inspectors judge and return exactly one verdict per run. The built-in inspector is a deterministic, stateless rules inspector. The four verdicts:

• allow — forward the response to the agent unchanged (byte-identical, including on the approved path).
• redact — forward a replacement body instead of the original (v1 covers JSON bodies only; a redact on an SSE stream fails closed with an error rather than rewriting the stream mid-flight).
• reject — refuse the response outright; the agent receives an RFC 9457 problem it can react to.
• escalate — hold the response and wait for a human decision (the approval loop below). This is the human-in-the-loop verdict.

When several inspectors run, the pipeline resolves to the most severe verdict: reject beats escalate, escalate beats redact, redact beats allow. A reject in one inspector short-circuits the rest.

Authoring inspection policy

Inspection policy is the llm.inspection governance key. It resolves for each agent through the same scope chain as every other LLM governance key, and an absent key means the agent is ungoverned: the pipeline is inert.

The value is a JSON document listing ordered inspectors and each inspector’s rules. A rule matches a tool-call finding by tool-name pattern (glob, e.g. transfer_usd or mcp*) plus optional numeric-argument predicates, and carries the verdict a match produces. The first matching rule wins:

{
  "inspectors": [
    {
      "name": "payments",
      "rules": [
        {
          "tool": "transfer_usd",
          "arg_gt": { "amount": 1000 },
          "verdict": "escalate",
          "summary": "transfer over the $1,000 threshold needs operator approval"
        }
      ]
    }
  ]
}

An arg_gt predicate holds when the named argument is strictly greater than the threshold: a transfer of exactly 1000 does not match a 1000 threshold. A rule with "verdict": "reject" refuses matched calls outright instead of holding them. A malformed policy document fails closed: the governed call is refused rather than a partial policy being applied.

Write the key with dvx policy llm inspection, which reads the JSON from --file (- for stdin). Scope flags select the target exactly like the other dvx policy commands: no flags is platform scope, --org, --group, and --agent narrow it, and --lock pins the key so more-specific scopes cannot override it:

$ dvx policy llm inspection \    --controlplane https://cp.example.com \    --org org_acme \    --file payments-inspection.json

The enclave picks the change up on its next policy snapshot pull, so a governed agent becomes governed (or the reverse) without restarting anything. See Author and pull policy for the scoping flags, locks, and the pull mechanics in full.

Operating the approval loop

When a rule escalates, llmproxy holds the response and registers a pending approval with the enclave broker. The held response waits there; the agent receives nothing while the hold is open.

Work the loop from the CLI. These commands talk to the broker’s operator API (directly at the broker, or through the relay), authenticate with the shared operator flags (--zitadel-key + --issuer, or --token-file, plus --ca-cert), and take --broker-url:

$ dvx approvals list --broker-url https://relay.example.com:8452 ...b41a1f9e-...  pending  tool=transfer_usd  agent=agent-a  args: {"customer": "acme", "amount": 5000, "currency": "USD"}

list shows pending and recently resolved approvals with the tool name, arguments, and agent, which is what an operator needs to decide. get <id> shows one approval. Decide with approve or deny:

$ dvx approvals approve b41a1f9e-... --broker-url ...$ dvx approvals deny b41a1f9e-... --broker-url ...

Approve and the held response reaches the agent, which proceeds with the tool call exactly as if no hold had happened (the forwarded bytes are identical to what the model sent). Deny and the agent receives an error instead, described below. Deciding an already-decided or expired approval is a conflict error; the original decision stands.

For a web UI or dashboard, the broker exposes the same feed as server-sent events at GET /api/v1/approvals/stream: each pending registration, decision, and expiry arrives as it happens, in order, with a heartbeat on idle so the connection’s liveness is observable. It is org-scoped bearer-authenticated like every operator route, so it is safe to proxy to a browser client. dvx approvals stream tails the same feed in a terminal until interrupted. See Operator API for the authentication model these routes share.

The disclosure toggle, timeout, and what the agent sees

Two operator-configured behaviors shape the agent-facing outcome of a denial.

The disclosure toggle (--inspection-disclosure on dvx serve llmproxy, default detailed) controls how much of the reason the agent sees:

• detailed — the agent’s problem names the reason: “tool call denied by operator” on a deny, “approval timed out” on a timeout, or the rule’s own summary on a reject.
• opaque — the agent gets only “request denied by policy”, on every denial path (reject, deny, and timeout alike).

Operators always see the detail either way: the approval list, the SSE feed, and the audit events carry the full reason in both modes. The toggle governs only the agent-facing error.

The timeout (--escalation-timeout on dvx serve llmproxy, default 5 minutes) bounds how long a held response waits for a decision. A hold that passes its deadline is treated as a deny, the approval row becomes expired, and the agent receives the timeout error. Silence never resolves to an allow.

For each outcome, the agent sees:

• allow (or an approved escalation) — the model’s response, forwarded unchanged; the tool call proceeds normally.
• reject — a 403 problem (detailed mode names the rule’s reason).
• escalate, then deny — a 403 problem (“tool call denied by operator” in detailed mode).
• escalate, then timeout or expiry — a 504 problem (“approval timed out” in detailed mode).
• redact on a stream — a fail-closed error; v1 does not rewrite streams.

Every denial is a structured RFC 9457 problem, so a well-behaved agent can adjust its next step instead of hanging. The fail-closed rule is uniform: a deny, a timeout, a broker outage, or a policy error all refuse the tool call; nothing ever falls open.

Try it end to end

Pair this guide with Run an agent against a local enclave: govern the agent with an inspection policy, give it a transfer tool, and watch an over-threshold call surface as a pending approval in dvx approvals list while an under-threshold call flows through with no approval step at all.