Human-in-the-loop: which AI agent actions need approval

Why do some actions need human approval?

Because an agent can be manipulated by the text it reads, and a human is the last control that is independent of the model. OWASP phrases it for LLM06:2025 (Excessive Agency): use human-in-the-loop control to require a human to approve high-impact actions before they are taken. The same page also recommends complete mediation, meaning authorization enforced in downstream systems rather than left to the LLM's judgment.

The EU AI Act (Article 14) points the same way for high-risk systems: human oversight proportionate to the risks, level of autonomy and context, with the ability to disregard, override or stop the system, and vigilance against automation bias. These obligations apply from December 2, 2027 for Annex III systems.

Human approval is therefore both a security measure and, in some cases, a compliance requirement.

How do you rank actions by risk level?

With three questions: is the action reversible, what can it touch, and who bears the consequences. The graver the answer, the stronger the control.

The level depends on context. An internal email to yourself is level 1. The same email sent to a customer list is level 2 or 3. So assess the action with its arguments (recipients, amount, volume) and not by the tool name alone.

Which actions always require approval?

Those where a mistake or manipulation is costly and cannot be recovered. Here is the starting list we use to review an agent.

• Money: payment, refund, change of bank details, purchase order creation.
• Outbound communication: emails, messages and posts addressed to third parties, especially in volume.
• Deleting or overwriting data without a restorable backup.
• Rights and identities: user creation, role change, sharing a file externally, issuing tokens.
• Data export to a domain, account or service that is not already approved.
• Code or command execution outside a sandbox: shell, write SQL, deployment.
• Changes to its own configuration: allowed tools, approval rules, MCP servers.

The last point is far from theoretical. In CVE-2025-54135 (Cursor), the agent could create an MCP configuration file without approval, and in CVE-2025-53773 (Copilot and VS Code), an injection could switch on auto-approval mode in the settings. In both cases, human approval was bypassed by editing what governs it. See our article on coding agent security.

How do you design an approval that actually protects?

By having the system enforce the approval, on the real action, with a presentation that lets a person decide. A badly designed approval gives a false sense of security.

1. Enforce server-side. The tool refuses to run without an approval token bound to the exact action. The model cannot decide that an approval is "already given".
2. Show the real action. The human sees the effective parameters: recipient, amount, SQL query, file. A summary written by the model can be manipulated.
3. Bind the approval to the action. If the arguments change after approval, a new approval is required. An approval is short-lived and single-use.
4. Identify the approver. A named person with the required rights, distinct from the requester when the stakes justify it.
5. Log the request, what was shown, the decision and the timestamp. See what to log.
6. Provide a simple refusal and a way to stop the agent.

What pitfalls make human approval fail?

The main one is approval fatigue: if a human approves dozens of benign requests, they end up clicking without reading. The others are more mechanical.

• Too many requests. Reserve approval for levels 2 and 3, and automate levels 0 and 1 with limits. A rare approval is an approval that gets read.
• Misleading presentation. Long text with the key element buried, a shortened link, a truncated recipient.
• "Always allow" option. It turns a one-off approval into a permanent authorization, including for an action triggered by an injection.
• Interface-only approval. If the tool's API can still be called without a token, the control is worthless.
• Automation bias. The human trusts the agent because it is often right. Article 14 of the EU AI Act asks precisely to stay alert to this bias.
• Bypass through content. The agent is led to rephrase a dangerous action as a series of small actions, each under the threshold. Assess call sequences as well.

How do you test that human approval holds?

By trying to bypass it with harmless payloads, from the viewpoint of an attacker who controls text the agent reads. The basic tests:

1. Direct call. Call the level 3 tool through the API without an approval token: it must be refused.
2. Substitution. Get approval for a benign action, then change the arguments: execution must fail.
3. Splitting. Break a sensitive action into level 1 steps and see whether a cumulative limit exists.
4. Injection. Put in a document an instruction asking to approve or skip the step, and verify it has no effect.
5. Self-modification. Ask the agent to change its approval rules or create a tool without approval.
6. Readability. Show the approval screen to a third party: can they say what will happen in ten seconds?

These are classic scenarios of our work on tool calls and MCP. The full framework is in our AI agent security pillar.

Where do you start this week?

List every tool your agent can call, give each a level from 0 to 4 using the table, and look at levels 2 and 3 first. For each one, check three things: is approval enforced server-side, does the approver see the real parameters, and is the decision logged. Fixing those three points on your five riskiest tools removes most of the exposure.

How does CleanIssue review an agent's actions?

We map the tools, assign a level to each action with its real arguments, then try to get past the controls. The Agent Red Team Express at €1,900 delivers in 5 days the ranking of your agent's actions and any bypass evidence. If several agents are involved, the Custom Red Team is quoted based on scope. The first conversation is a free 30-minute discovery call.