Why Agents Fail on Complex Software

An agent that scaffolds a working SaaS demo in twenty minutes will, on the same afternoon, quietly break invoice idempotency in a five-year-old codebase and hand you a green test suite. This is not inconsistency. Greenfield toys and complex systems are different problems, and agents are strong at exactly one of them. Every failure mode below is predictable — which means preventable.

Task execution vs engineering judgment

Agents are excellent at task execution: given a well-defined transformation, produce the code. They are weak at engineering judgment: knowing which transformation is safe here, what the change will break two modules away, and when to stop and ask. A greenfield toy is almost pure execution — no history, no invariants, no users. A production system is mostly judgment with execution sprinkled in. When you delegate a task, you're delegating the execution; the judgment either comes from you (in the spec) or gets improvised by the agent (badly). See Responsibility Split.

Missing invariants: the rules that live in heads, not repos

Every mature codebase has load-bearing rules that appear nowhere in the code:

• "Orders can go pending → paid → shipped, never backwards — support once manually flipped one and it double-charged."
• "The users.email column has case-sensitive collation for legacy reasons; always compare lowercased."
• "Service B silently depends on service A's JSON field order because of a bug we never fixed."

A human absorbs these through incidents, review comments, and hallway lore. The agent gets none of that — it sees only what's in the repo and its context window. It will violate unwritten invariants at exactly the rate you'd expect from a brilliant contractor on day one. The fix is structural, not prompt-side: invariants must live in the repo. See Docs as Memory and CLAUDE.md and AGENTS.md.

Hidden coupling

Agents reason locally: read the relevant files, change them coherently. Complex systems fail globally: the cron job that parses this log format, the mobile client pinned to this response shape, the downstream ETL that assumes this enum has four values. If the coupling isn't discoverable by reading nearby code — and the worst coupling never is — the agent can't account for it. A change can be flawless in every file it touched and still cause an incident in a repo it never opened.

Plausible ≠ correct

Agent output is optimized to be convincing: idiomatic, well-named, commented, typed. Human reviewers use surface quality as a proxy for correctness because for human authors the two correlate — sloppy code usually means sloppy thinking. Agents break that correlation. They produce immaculate-looking code containing a subtly wrong currency rounding rule, with the same confidence either way. Review AI diffs for behavior against intent, never for craftsmanship — craftsmanship is free now and signals nothing. This is the core skill in Reviewing AI PRs and the theme of Hallucination and False Confidence.

Local coherence vs global consistency

Within one task, agents are highly coherent. Across tasks and sessions, they have no memory and no taste continuity: session 1 handles errors with a Result type, session 3 throws exceptions, session 7 introduces a third pattern — each locally defensible. Twenty sessions later you have a codebase with five conviction systems and no convictions, and every future session gets worse because the agent now sees contradictory precedents and picks one at random. Architectural consistency is a human responsibility enforced through repo docs and review; it does not emerge from good prompting. See Architecture Erosion.

Vague prompts create shallow software

Agents resolve ambiguity toward the median implementation — the statistically typical way the feature is built in public code. Median code handles the happy path, ignores concurrency, skips the empty state, and hardcodes the config. Your product is presumably not the median; the entire value of your requirements lives in the deviations. A vague prompt discards exactly those deviations and hands you a demo-quality implementation of someone else's average product. Depth in, depth out. See Shallow Implementation and Prompt Anatomy.

Silent assumptions: a worked example

The prompt:

Add rate limiting to the API.

Six decisions the agent now makes without telling you:

1. Algorithm — fixed window? sliding window? token bucket? It picks one; they behave differently under burst traffic.
2. Scope of the limit — per IP? per API key? per user? per endpoint? Per-IP silently rate-limits every customer behind one corporate NAT.
3. State storage — in-process memory is the easy default, and it breaks the moment you run two instances behind a load balancer.
4. Rejection behavior — 429 with Retry-After? Queue? Drop? Does the mobile client even handle 429?
5. Exemptions — health checks, internal service calls, webhook deliveries, and your own admin panel are now rate-limited too.
6. Configuration — the limit is hardcoded at whatever number appeared most often in training data. 100/minute, probably.

The agent ships a clean middleware, tests included, all green. Three of those six defaults are wrong for your system, and you discover which three in production. The prompt was one sentence; the feature was six decisions. The gap between them is where agents fail — and it's exactly what a spec exists to close. See Prompts vs Specs and Writing Effective Specs.

Failure table

Failure	Symptom	Root cause	Early detection	Fix
Missing invariants	Change violates a rule "everyone knows"	Rule lives in heads, not repo	Ask agent to state assumptions before coding	Write invariants into CLAUDE.md / docs
Hidden coupling	Green tests, broken downstream consumer	Coupling not visible from touched files	Require a blast-radius list in the plan	Document cross-system contracts; integration tests at boundaries
Plausible ≠ correct	Polished diff, subtle behavioral bug	Fluency uncorrelated with correctness	Review behavior vs intent, not style	Executable acceptance criteria; see Verification Strategy
Consistency drift	Three error-handling patterns after ten sessions	No cross-session memory	Diff new patterns against documented conventions	Conventions in repo docs; reject nonconforming diffs
Shallow implementation	Happy path only; edge cases missing	Vague prompt → median code	Grep the diff for the edge cases you named — or didn't	Spec the deviations from median; see Shallow Implementation
Silent assumptions	Built the wrong variant of the right feature	Ambiguity resolved without surfacing	"List open questions before implementing"	Decide the decisions up front in a spec
Runaway confidence	Agent asserts done; it isn't	No self-verification pressure	Demand it runs the proof, not claims it	Quality gates; see Quality Gates

The takeaway

Agents don't fail randomly on complex software — they fail where the system's real definition lives outside the repo and outside the prompt. Every fix above is the same move: relocate judgment, invariants, and intent into artifacts the agent can read. That move is the subject of the entire specs section and the failure-modes catalog.

Related

• Mental Models — the compiler-for-intent model that predicts every failure on this page.
• Prompts vs Specs — closing the six-silent-decisions gap with durable specs.
• Failure Modes — the full catalog with per-failure detection, prevention, and recovery prompts.
• Shallow Implementation — the vague-prompt → median-code mechanism in detail.
• Docs as Memory — getting head-resident invariants into the repo.
• Reviewing AI PRs — review technique for plausible-but-wrong output.