Agentic Engineering

Appearance

Shallow Implementation ​

Agents resolve underspecified work toward the median public implementation: happy path, typical SaaS defaults, demo-grade edge handling. Your product's value usually lives in the deviations from that median. When those deviations are not in the prompt or spec, you get something that demos well in a screenshot and fails in production. Three failure modes share this root; treat them as a cluster.

1. Shallow demo instead of real system ​

Symptoms ​

  • Happy path works in a local click-through; empty states, errors, concurrency, and permissions are stubs or missing.
  • "TODO" / hardcoded config / in-memory stores left where production needs persistence and ops hooks.
  • README claims features the code only sketches.
  • Stakeholders say "looks great"; on-call will disagree within a week.

Root cause ​

Vague goal ("build invoicing") with no acceptance criteria, non-functional requirements, or out-of-scope fence. The agent optimizes for visible completeness — screens and routes — not for load-bearing behavior. See Why Agents Fail and Prompts vs Specs.

Early detection ​

  • Grep the diff for the edge cases and ops concerns you care about; absence is the signal.
  • Ask: "What happens when X fails?" If the answer is only in chat, not in code/tests, it is a demo.
  • Check for real persistence, authz on every mutating route, and idempotency where money/state is involved.

Prevention ​

  • Spec acceptance criteria as observable behaviors, including failure paths.
  • Explicit quality bar: "no in-memory stores; no TODO in src/; every write path has authz."
  • Tester-first for the non-happy paths (Patterns).

Recovery prompt ​

text
This implementation is demo-depth. Do not add features.
Against docs/specs/<feature>.md, produce a gap list:
1. Missing failure / empty / authz / concurrency paths
2. Hardcoded or in-memory pieces that must be real
3. Tests that only cover the happy path

Then implement ONLY the gaps needed to meet the spec's
acceptance criteria. No new scope. Paste test evidence.

Review checklist ​

  • [ ] Every acceptance criterion has a test or explicit manual proof
  • [ ] Error and empty states are real UI/API behavior, not placeholders
  • [ ] No TODO/FIXME in production paths for this feature
  • [ ] Config is externalized; secrets not hardcoded
  • [ ] Authz present on mutating endpoints

2. Easiest version instead of required version ​

Symptoms ​

  • Spec asked for sliding-window rate limit; agent shipped fixed window "for simplicity."
  • Required multi-tenant isolation; agent filtered by userId only and called it done.
  • "Supports CSV export" means a client-side download of the current page, not the full dataset.
  • Performance/scale requirements ignored in favor of the algorithm that fit in one file.

Root cause ​

Agents minimize implementation cost under ambiguity. "Required" that lives only in a conversation turn loses to "easy" that matches training priors. Latest-instruction pressure can also collapse a careful plan into a shortcut — related to overfitting the latest instruction.

Early detection ​

  • Diff the implementation against the named algorithm / constraint in the spec, not against "does something rate-limity exist."
  • Require the agent to quote the spec line each major module claims to satisfy.
  • Benchmark or property-test where "easiest" and "required" diverge (e.g. burst behavior).

Prevention ​

  • Put non-negotiables in the spec as constraints, not wishes: "must be sliding window; fixed window is incorrect."
  • Out-of-scope list that includes the easy wrong alternatives ("do not implement fixed-window").
  • Plan-mode approval of the approach before coding (Architect then builder).

Recovery prompt ​

text
Spec constraint: <quote the required behavior>.
Current code implements <easy variant>. That is incorrect.

Replace the easy variant with the required one. Do not keep
both. Update tests so the easy variant would fail. List any
spec ambiguities before coding — do not invent a third variant.

Review checklist ​

  • [ ] Named algorithm/constraint in spec matches code (not a cousin)
  • [ ] Tests fail if the easy variant is reintroduced
  • [ ] Comments/docs do not describe the required version while code does the easy one
  • [ ] No "simplified for now" without a tracked follow-up and flag

3. Mixes product assumptions with implementation ​

Symptoms ​

  • Agent invents pricing rules, roles, or workflows nobody approved ("admins can always refund").
  • Copy, status names, or email triggers appear from SaaS folklore, not from the spec.
  • Data model includes fields for features explicitly out of scope.
  • Product and eng argue about behavior the agent silently chose.

Root cause ​

Product decisions look like implementation details to a model trained on full apps. Ambiguity is filled with typical product, not with questions. Silent assumptions are the mechanism — see the rate-limit worked example in Why Agents Fail.

Early detection ​

  • Force "list open questions / assumptions" before coding; treat each assumption as a product ticket.
  • Review domain rules in the diff as if they were a PR from a PM who cannot code — would you approve the policy?
  • Compare enums, statuses, and permissions to the spec glossary; extras are inventions.

Prevention ​

  • Spec owns product rules; implementation prompts forbid inventing them (Writing Effective Specs).
  • CLAUDE.md: "If the spec is silent on a product rule, stop and ask — do not invent."
  • Separate product clarification session from build session (Idea → Spec).

Recovery prompt ​

text
Diff against docs/specs/<feature>.md. List every product rule
the code implements that is not in the spec (statuses, who can
do what, pricing, notifications, defaults).

For each: REMOVE it, or mark it BLOCKED pending human product
decision. Do not leave invented rules in place. Do not expand
the spec yourself — propose wording only.

Review checklist ​

  • [ ] No domain rule in code without a spec citation
  • [ ] Out-of-scope features absent from schema and UI
  • [ ] Defaults (limits, roles, retention) match spec or are flagged
  • [ ] Agent assumption list reviewed and either accepted into spec or reverted

A field manual for AI-native software engineering.