Continue? Y/N · Approval fatigue: when humans turn into robots

Continue? Y/N is a game with a simple premise: can you quickly spot the threats hidden in your agent's commands? You have 60 seconds to shine among the git status and npm test calls, or the occasional cat ~/.kube/config.

I played it, here's my card:

Continue? Y/N ── SCORE: 900
🛡️ Security-Conscious Engineer

🟩🟩🟩🟩🟩🟩
Caught 3/3 threats
"Not a single secret leaked"

→ llmgame.scalex.dev

Beyond the game, what it really puts a spotlight on is approval fatigue. A perfect awareness exercise for European Cybersecurity Month.

A safeguard that wears out

The phenomenon has a name: approval fatigue. The more checks you ask a human to make, the worse those checks get. They read less and less, and end up approving on autopilot.

For a CISO or a CIO, it's a harsh position to be in. After rolling out AI across the development teams, they still rely on those same teams for security. But the volume of requests becomes overwhelming, and even the best engineers end up approving without understanding.

An old acquaintance

Nothing new here: human fatigue is well known. In a SOC, it's alert fatigue. For end users, it's MFA bombing. In aviation, it's the checklist recited by rote. In law, it's consent fatigue.

On the psychology side, several mechanisms are at play: the low prevalence effect (Wolfe, Nature, 2005), the vigilance decrement (Mackworth, 1948), and the "cry wolf" effect (Breznitz, 1984).

Connectors are part of the picture too: as I wrote in my previous article, a rogue connector can go unnoticed by slipping its command between two harmless lines, and it's the tired user who signs off on it.

What to do about it

Paradoxically, the answer is to ask less in order to ask better. As in many systems, automated rules cut down the number of requests reaching the user. Clear instructions in an AGENTS.md can help, as can deterministic automation (hooks, for instance) or stronger isolation: Anthropic cut permission prompts by 84% with a sandbox. What's left to approve becomes rare, so it actually gets read.

Humans should be kept for what can't be undone: publishing, deleting, or sending data outside. Accountability for actions must not get diluted behind the AI.

What this means for a CISO or CIO

Human approval is not infallible. If a vendor or an internal team presents it as a security measure, it belongs in the risk analysis, along with its own mitigations for approval fatigue (number of requests, training for approvers, and so on).

Your teams' attention is a limited budget: better to spend it on the decisions that matter.

Further reading

• Continue? Y/N, Scale X, the game,
• How we contain Claude across products, Anthropic, on approval fatigue and sandboxing,
• Reframing LLM Agent Security as an Agent-Human Interaction Problem, arXiv,
• Rare items often missed in visual searches, Wolfe, Horowitz and Kenner, Nature (2005), on the low prevalence effect,
• The breakdown of vigilance during prolonged visual search, Mackworth, Quarterly Journal of Experimental Psychology (1948), on the vigilance decrement,
• Cry Wolf: The Psychology of False Alarms, Shlomo Breznitz (1984), on the "cry wolf" effect.