Not a benchmark · battle data

The Agent Failure Index

Every "agent reliability" number you've seen comes from a benchmark. This one comes from a single autonomous agent running in production — the gate firings it actually triggered, ranked by how often it tried to fail.

2,240
gate firings
55
distinct gates
21
failure modes
31
days

pip install "agent-contracts @ git+https://github.com/impartshadow/agent-contracts.git" · MIT

The headline finding: the most expensive failure mode isn't a dangerous tool call or a leaked secret — it's behavioral drift. The top two gates, accounting for 28% of all firings, both catch the agent repeating a behavior it was already told to stop, or claiming work was done with nothing behind it. Capability and security gates fire far less often than the gates that keep an agent honest about what it actually did.

Window: 2026-05-31 → 2026-06-30. Aggregate counts only — no prompts, no messages, no identifiers are published. Every row maps to an open-source, deterministic gate you can install today.

Failure modes, ranked by real firing count

IDWhat the agent tried to doGate that stopped itFiringsShare
FM-033
Repeated a stopped behavior
The agent re-did something the operator had explicitly told it to stop doing.
persistent-correction 638
28.5%
FM-027
Said 'done' with no artifact
A completion claim with no verifiable artifact — no commit hash, file, or live URL behind it.
completion-artifact 285
12.7%
FM-026
Claimed a fact with no evidence
A decision or assertion was made without reading the source that would confirm it.
partial-evidence-flag 257
11.5%
FM-011
Proposed instead of executing
The agent described or proposed an action it had the authority and tools to just do.
self-verification 154
6.9%
FM-029
Stated a fact without checking
A factual claim emitted from memory instead of a same-turn lookup.
factual-claim-verification 150
6.7%
FM-003
Looped without progress
The same file or action was hit 3+ times in a session with no forward movement.
loop-tripwire 121
5.4%
FM-023
Leaked a personal identifier
A name, email, or private identifier was about to go into an outbound surface.
dox-guard 108
4.8%
FM-004
Wrong tool route
A forbidden or non-canonical tool path was taken when a structured one existed.
web-tool-invocation-rewriter 106
4.7%
FM-022
Asserted stale state as current
Treated a cached or remembered value as the live one.
stale-state-assertion-guard 89
4.0%
FM-012
Told the human to do its job
The agent instructed the operator to do something it could have executed itself.
platform-action-precheck 87
3.9%
FM-014
Asserted ungrounded state
A claim about live system state with no same-turn check to ground it.
state-assertion-grounding 55
2.5%
FM-031
Skipped the pre-check phase
Acted before the required anticipation / pre-flight gate ran.
anticipation-phase-gate 42
1.9%
FM-015
Wrong destination
A push, send, or write was aimed at the wrong branch / recipient / path.
harness-scaffolding-leak-guard 29
1.3%
FM-017
Dangerous path write
A write targeted a system path (/etc, ~/, etc.).
dangerous-path-guard 28
1.2%
FM-NEW
FM-NEW
cl-stop_just_cold_including_stripe 28
1.2%
FM-025
FM-025
task-reframe-guard 21
0.9%
FM-013
Scope overrun
Did more than was asked — unsolicited side-trips or checks.
balar-clarification 19
0.8%
FM-005
FM-005
cl-channel_shadow_hq_system_bot 16
0.7%
FM-019
Unnecessary hedging
Hedged or deferred where a direct action was warranted.
behavioral-haiku-guard 4
0.2%
FM-001
Capability denial
Claimed it couldn't do something before attempting it.
behavioral-haiku-guard 2
0.1%
FM-024
FM-024
propguard-provenance-detector 1
0.0%

If you're shipping an autonomous agent, this is the distribution of ways it will try to embarrass you — measured, not guessed. The gates are MIT-licensed →