Two failures that look like competence

You typed a prompt, hit enter, and the AI confidently wrote code that imported a package that does not exist. Here is the machinery that produced that — and the exact controls that would have caught it.

“Writes code with complete conviction — including bugs or nonsense — and won’t tell you something is wrong unless you catch it.”

What an LLM actually does (and doesn’t)

Think of the model as a fluent improviser who never breaks character: it always produces confident, plausible output — with no internal fact-checker. Here is the four-step pipeline behind every response.

import faiss_cpu_optimized   # looks real, is not on PyPI

Every turn resends everything

The context window is everything the model can see this turn — system instructions, your rules files, the conversation so far, files you pulled in. What is not in the window does not exist for that turn.

Prompt caching saves the computed state of stable content so only new messages are processed fresh — cutting latency and cost. But it silently resets on any of these:

Four ways context goes bad

More context is not always better. A lean, relevant window beats a bloated one — here are the four failure modes to watch for (Drew Breunig, 2025).

You didn’t send a prompt; you started a loop

“My preferred definition of an LLM agent is something that runs tools in a loop to achieve a goal.”

Reframe it: not “I sent the model a question” but “I started a loop, and I can intervene at each of these four stages.” Step through the agent loop below.

Both incidents from the first screen broke down in Stage 3. The fabricated install and the destructive database command both happened when a tool ran without a gate or a scope to stop it.

The controls — and a 3-question diagnostic

The harness is the director and stage crew around the improviser: it is how you add control. Each loop stage exposes its own levers.

“If you give an agent a way to verify its work, it more often than not fixes its own mistakes and prevents regressions.”

Here is a post-tool hook that catches the “invented method” class automatically — it re-runs your type-checker after every edit.

// settings for your harness — run the type-checker after each edit/write
{ "hooks": { "PostToolUse": [
  { "matcher": "edit|write", "command": "run-your-typecheck" }
] } }

Check your mental model

Three diagnostic scenarios. Pick the fix a working engineer would reach for.

Next: Module 2 — Research → Plan → Implement, the operating pattern that puts these controls into practice.

Same prompt, two very different pulls

The prompt "Add anchor links to the titles in the topics list page" was run twice by the same model. Same words, same code, two unrelated results.

That inconsistency isn't the model being dumb. It's the workflow being absent — and here's the mechanism behind it.

"GitHub Copilot provides better results when assigned clear, well-scoped tasks." — GitHub best practices, 2026

Building a house, not pouring concrete and hoping

You survey the land, draw blueprints a contractor can build from, then construct. You don't pour the foundation and then decide where the kitchen goes. Three phases, three distinct jobs.

Research is questioning, not outsourced thinking

"The goal is to build a mental model of the problem space — not to produce output." — AgentPatterns, 2026

Research means asking follow-up questions until you understand, studying what already exists before proposing anything. A research prompt looks like a question, not a request for code.

How does authentication work in this app?

The blueprint is a "Done when" list

A plan's core artifact is a list of checkable outcomes you can review before any code exists. Here's a real one.

## Fragment Anchors Plan
Done when:
- Hash links work for page titles in the topics list
- Anchor IDs follow a consistent kebab-case naming pattern
- Deep links survive hash-router navigation
- No scope creep to other pages or unrelated features

Grounding changes the answer. The same auth prompt produced materially different plans with vs. without evidence attached.

With good blueprints, construction is mechanical

"Implementation becomes mechanical when the research and plan are solid." — AgentPatterns, 2026

Each implementation prompt cites the plan plus the work item, so the model's job is execution, not re-architecture. Quality gates run automated-first, then human review against "Done when".

The cost of being wrong is asymmetric. Where the error lives decides how much it hurts.

An AI plan is cheap. The wrong plan is expensive. Verify before you execute.

The handoff: passing blueprints between crews

Run each phase in a fresh session and carry a handoff artifact between them. Watch a research-to-implement handoff play out.

A good handoff is Distilled (conclusions only), Validation-carrying (the criteria that prove the next step worked), and Durable (survives compaction — re-loaded next session). If implementation fails, update the plan and retry. No need to re-research.

Scaling up: when the work is too big for one plan

Most product requirements don't arrive pre-broken-down for an agent. Extend the flow with two moves before you build.

A vertical slice is one behavior end-to-end; its tasks are the dependency-ordered build steps (model → schema → API → UI). Write the acceptance criteria as Given/When/Then.

Review this task breakdown for gaps and risks. For each: what could go wrong,
wrong assumptions, conflicts with existing architecture, scope creep, and per-task
risk (low/med/high). Flag any story likely over 2 days and suggest how to split it.
Focus on what could go wrong, not what could go right.

Check your instinct

No definitions here — just judgment calls. Pick what a disciplined engineer would actually do.

Your new colleague's first day

An agent that keeps asking the same question, retries the same command, or edits the wrong folder isn't broken — it's under-briefed. Everything it needs to succeed is something you can set up once.

Picture onboarding a new hire. Leave them the right setup and they ship in an hour; leave them nothing and they spend the day guessing. An agent is that new hire every session — with no memory of yesterday unless you wrote it down.

The permission mode is your oversight dial

Before rules and docs, know the one control you touch every session. The permission mode sets a ceiling on what the agent may do without stopping to ask. The names differ by tool; the four categories are the same.

Match the mode to what you want to pause for — friction is a dial, not a personality. In most tools a single cycle key (like Shift+Tab) rotates through the modes.

Rules capture the correction you keep repeating

If you're telling the agent the same thing twice — "use our logger, not raw print statements", "tests live in a dedicated folder", "don't touch the legacy billing code" — that repetition is a signal. Capture it as a durable rules file such as AGENTS.md or CLAUDE.md.

The rules lifecycle — step through it

A rules file is one member of a family of durable configuration you can leave behind:

Write the behavior, not the intent

A rule the agent can act on names the exact behavior. Vague intent forces it to guess — and it guesses differently every time.

What earns a place — and what doesn't

One more thing worth writing down explicitly: what the agent is allowed to read. That is controlled by the tool's ignore or deny config — not by .gitignore, which only affects discovery, not agent access.

# in your agent config
permissions:
  deny:
    - "legacy/billing/**"
    - "**/*.env"
# or, tool-specific: .cursorignore
legacy/billing/
*.env

Docs that pass the fresh-agent test

Judge an onboarding doc by one test: could a fresh session build and run one test using only this doc, in under ten minutes? Four moves measurably lift output — the deltas are from a study across 2,500+ repos.

Two anti-patterns that look like "more help" but hurt

Both backfire for the same reason: frontier models reliably follow only ~150–200 instructions. As the count climbs, adherence degrades across all of them. Every line competes for the same budget.

Short root, progressive disclosure, pointers not copies

Structure the docs so the agent loads only what a task needs. A short root file acts as a table of contents; progressive disclosure keeps each topic in its own file.

# AGENTS.md
- This service uses pnpm, not npm.
- Build commands live in package.json
  scripts — see there, do not duplicate.
- Never modify legacy/billing/ —
  pre-2022, business-critical.
- If you are touching payments,
  read agent_docs/payments.md first.

And keep docs living: any PR that changes a build, convention, or deploy step updates the doc in the same diff. Flag any doc unchanged for 90 days while the repo kept moving.

A deterministic build is a contract

"Local green is not enough." A build is deterministic only when a teammate, CI, or an agent can reproduce it from a clean checkout. Environment drift is why the same clone passes on two laptops and fails in CI. More discipline won't fix it; a process will.

Different artifacts for different consumers

The most common silent blocker is a private registry: name the feed URL and the auth step in your docs — even if it's just "ask this person for the credential."

Check your instinct

Four situations. Decide what a set-up-to-win engineer does before you reveal the answer.

The counterfeit that passes at the counter

A cashier catches a crude fake in a glance. What gets through is the good counterfeit — right paper, right ink, matching the real bills around it. AI code fails the same way: it compiles, passes lint, reads like its neighbors, and is still wrong.

The volume problem

Faros AI, 22,000-developer dataset, 2025. Volume went up; review capacity didn’t. So your review time has to become effective, not performative.

The five ways a counterfeit gets through

Every AI failure you review falls into one of these five families. Learn the tell for each and review stops being a vibe and becomes a checklist.

Vacuous tests, up close

The most seductive counterfeit: a green test suite that proves nothing. Here a payment test looks thorough and asserts nothing real.

charge = jest.fn().mockResolvedValue({ ok: true });
processOrder(order);
expect(charge).toHaveBeenCalledWith(order.total);

Its siblings — same trick, different disguise

Spot the counterfeit

This is plausible-but-wrong logic in the wild. It reads like tidy cleanup and matches the style around it. One line silently destroys valid data. Click the line you’d flag.

Don’t inspect every bill equally

Sustained vigilance across a whole diff fails — attention runs out before the lines do. Tier the diff first, then read the high-value lines closely.

Driving a ticket: rig the checks before the writing

You don’t just review AI code — you drive tickets end-to-end. The daily loop:

An “agent-sized” ticket fits one PR (target under ~400 lines changed), has acceptance criteria the harness can verify, and needs no simultaneous migration or cross-service deploy to test locally.

# AGENTS.md
Before saying you are done: run `./scripts/check`.
If it fails, fix and re-run.

You sign for every claim in the PR

A PR template tuned for AI diffs — one that lives on the source branch of your code host — turns the vague “looks good” into five targeted defenses:

## What & why — one paragraph + linked ticket
## Test plan
- Happy / Error / Edge: one concrete case each
## AI authorship
- [ ] Tools used: <which>
- [ ] I read every line of this diff myself
## Reviewer focus
- Read closely / Sample / Trust-signal: <files>
## Pre-flight
- [ ] New imports verified (no hallucinated packages)
- [ ] Searched repo for existing utilities

Check yourself

Four situations from the daily loop. Pick the move a calibrated reviewer makes.

Cheap levers beat heroic review

Picture a trapeze artist working above a safety net. The performer is non-deterministic — even a great one slips. The net is deterministic: same spot every time, it never has to recognize the fall, it just catches.

Formatter vs linter: determinism is the product

Two tools, two jobs. Running one through the other is the classic anti-pattern — keep the layers separate.

The right gate at the right stage

# CI: verify only — never rewrite files
prettier --check .
eslint .

# Agent post-edit hook: format every file it writes
on_file_write: prettier --write $FILE

Paying down lint debt: the ratchet

You want a strict new rule. The codebase has 3,300 existing violations. Three ways forward — only one ships fast without blocking everyone.

Testing: draw the line, keep the harness deterministic

The test pyramid tells you how to spend your testing budget — most tests fast and low, a few broad and slow.

AI-assisted test generation is a four-phase loop

Flaky tests: own them, don't re-run them

A flaky test passes and fails on the same code. At scale the noise is enormous.

Properties of a high-signal test

Triage decision rules

A smoke test is a behavioral assertion — not /health

Improve one piece of the net each iteration (n+1 > n). Since the AI ships more code than review can catch, the post-deploy net has to absorb the difference.

# What most pipelines call a "smoke test"
curl https://api.example.com/health
# -> 200 OK  -> "deploy succeeded"

# An actual behavioral smoke test
POST /checkout {cart} -> 200 {order_id: "…"}

Seven shapes exist — API contract, user journey, integration, data-shape, background job, agent/LLM path, and a negative permission check that asserts an unauthorized request is still rejected.

The hero move: canary deploy → auto-rollback

A canary limits the blast radius: a bad version only touches a sliver of traffic before the gate stops it.

Rollback, reversibility, and safe schema change

Five recovery moves exist — but only two are true rollbacks. Know which lever you are actually pulling.

None of these undo migrations that already ran, rows the bad code wrote, or emails sent — those need a compensating action.

Safe schema change: expand → migrate → contract

Two services on their own deploy schedules still share one schema — a shared database still couples them. Never drop or rename a column in the same deploy that removes the code using it.

This is why a schema migration is a one-way door: the migration has already run, so you sequence it in stages instead of relying on rollback.

Check your instincts

Four situations. Pick what a senior engineer would actually do.

When is an agent dangerous?

Even with nobody attacking it, an agent that can act — deploy, delete, change data — can do real damage by an honest mistake. It gets far worse when three things line up.

With all three, a booby-trapped page can quietly turn the agent against you — because the model can't reliably tell instructions from content. This is prompt injection, and it echoes the OWASP Top 10 for Agentic Applications (2026). Mindset: build like it will go rogue — not because it usually does, but because the cost when it does is too high to leave to luck.

Watch the guardrail fire

A support ticket carries hidden instructions. The agent can query the database and run commands — the trifecta is live. Play it through and watch the approval hook stop a disaster.

The agent believed the ticket. The hook didn't — it stopped an action, not an intention. That is a guardrail. A politely-worded "please don't" would not have saved you.

Two ways to set a limit — plus a layer underneath

Onboarding metaphor: would you give a new hire production database access on day one? You wouldn't — not from distrust, but because one honest mistake costs too much. You scope access to the task. An agent is that new hire.

Why the sandbox matters: lists and hooks depend on recognizing the action — a rephrased command can slip past either. The sandbox recognizes nothing; it just blocks the syscall. Here is what a standing limit looks like in a settings file:

{ "permissions": {
    "allow": ["Bash(npm run *)", "Bash(git diff *)"],
    "deny":  ["Bash(rm *)", "Bash(git push *)",
             "Bash(curl *)", "Read(.env*)"]
} }

Least privilege is the whole game

What goes wrong → the guardrail that catches it

Why big files hurt agents specifically

A god file is a wall you can't tell is load-bearing. Knock it down blindly and the ceiling comes with it.

Find the seams, lock behavior, then move

A seam is a place you can alter behavior without editing at that exact spot. Each seam has an adaptor: constructor injection, a factory, a function parameter, a link-time stub. Classic case: code calls Clock; a seam lets you slot in a FakeClock for tests without touching the call site — that's dependency inversion.

Keep the model on judgment, not bulk edits, with a read-only research pass — the same Research → Plan discipline from Module 2, aimed at a scary file:

Read this file. List every top-level function or
class and assign each a single responsibility:
{data access, validation, business rule,
formatting, I/O, framework glue, dead code}.
Propose a target module tree, one responsibility
per module. Flag any symbol that belongs in two
places — those are your decomposition targets.
Do not move anything yet.

Ship it in small, reversible slices

Pick the strategy by risk. Every slice is a small PR — ≤ ~300 lines, revert-safe (no mid-sequence data migrations), all tests green (including characterization) before merge.

Six pitfalls to watch

Check your judgment

Four scenarios. Pick the move a careful engineer would make.

Tool vs. skill

You keep re-pasting the same five-step procedure into the agent. It works, but it lives in your chat history where a teammate can never find it, and stuffing it into an always-on rules file makes every session pay for it.

A tool is a raw ingredient in the pantry. A skill is the written recipe that turns ingredients into a dish. Fifty recipes in a drawer cost nothing until you open one.

A skill uses tools — it's the layer above them, not an alternative. Skills don't give the agent new powers; they give it expertise, loaded on demand so 50 skills cost nothing until one fires.

Anatomy of a skill

Here's a real, tiny SKILL.md. Every part earns its place — the top block is metadata, the bottom is the recipe itself.

---
name: summarize-changes
description: Summarizes uncommitted changes and flags
  anything risky. Use when the user asks what
  changed, wants a commit message, or reviews a diff.
allowed-tools: Bash(git diff:*)
---
## Current changes
!`git diff HEAD`
## Instructions
Summarize the changes above in two or three
bullets, then list any risks. If the diff is
empty, say so — do not guess.

The description is the whole ballgame

Among 100+ installed skills, the description is the only thing the model reads to decide. Get it wrong and a perfectly good skill never fires. The formula: what it does + when to use it + key capabilities.

Be pushy with triggers. Models under-trigger far more than they over-trigger, so an explicit “Use when…” beats a subtle implication every time.

Two authoring habits that pay off

Keep it small, then author in a loop

Progressive disclosure: the SKILL.md stays tiny; detail lives in bundled files that load only when the task needs them.

Three levels: metadata (always in context) → body (loads when triggered) → bundled files (read or run only when pointed at). That's why a skill can be thorough without being heavy.

Author in a loop

A skill is never right on the first draft. Build it, run it on a real task, then read what the agent actually did — and go again until the runs stop surprising you.

When not to build a skill

“A skill that is only fixed steps is a script wearing a costume.”

A saved query beats a raw log dump

When something breaks, the agent dumps 200 lines of logs into its context and loses the thread. Your logs are a patient's chart: raw notes are a shoebox the agent reads cover-to-cover and still misattributes; a clean, indexed chart lets it find the cause in seconds.

# BEFORE — raw log dump (~200 lines, ~12,000 tokens)
2026-05-19T14:02:11Z svc=checkout ...
svc=payments ... ERROR svc=auth
trace=7f3a48 Read timed out ...
 
# AFTER — one saved query (~600 tokens, same answer)
SELECT service, COUNT(*) AS error_count
FROM logs WHERE status = 'error'
GROUP BY service ORDER BY error_count DESC LIMIT 10
→ auth-service 184 | payments-api 42 | checkout-api 19

Four improvements that make data agent-readable

You don't need a new platform — four small changes turn raw output into evidence the agent can reason over.

The keystone is structured logging — here's the whole idea in two lines:

// Before
console.log('user 123 paid');
 
// After
logger.info('payment_received', {
  tenant_id: '123', trace_id: traceId,
  amount: 4250 });

Lock it down before you let it loose

Once an agent can read your systems, two things matter: what it can touch, and what it can see. Read-access lockdown has three tiers — and only one is a real boundary.

Classify before the agent reads

One skill fires three ways

Check your instinct

Four situations. Pick what you'd actually do, then check.