How Much Planning Does a Coding Agent Need?

On July 15 at 13:28, a design task on my board for Maestro opened with a locked direction: "Managing user databases is not Maestro's job." The direction was mine. The wording, and the sketch of mechanics under it, came from the coordinating agent I run my work through. The sketch was specific: a scheduler notices a stopped session past its retention window, emits an archive signal, the machine running the session executes the project's archive hook and confirms, and the backend stamps the session as archived.

A worker agent claimed the task four minutes later. At 13:52 it delivered a design document with a 22-row table that checked every claim the task leaned on against the current code. Four of the sketch's facts were wrong. One of them would have re-archived every database every 14 days, forever.

That plan earned its twenty minutes. The tempting conclusion is to put a plan in front of every ticket, and most coding agents now make that easy: plan mode, spec tools, a grooming step. By grooming I mean the pass where an agent reads the code and docs and commits a plan before it writes any code. I've also run two studies on my own harness that ask the per-ticket version of the question, and they point somewhere else.

I wanted to know which work deserves a plan, and what I should look for when an agent hands one back. The short answer to the second: judge a plan by what it found, not by how long it is.

Four Facts Wrong in a Confident Sketch

The task was written at 13:20 and rewritten at 13:28 to drop every carve-out. Then the worker read the code, and four facts in the sketch didn't survive:

1. There was no acknowledgement channel. Signals lived in an in-process queue and vanished when nobody was subscribed, so nothing could "confirm" the way the sketch assumed. The design made the 60-second sweep itself the retry, because a lost signal re-derives from durable state on the next tick.
2. The archive hook had never run. It sat in the config, but no runtime path executed it.
3. The signal had to carry the project name. Hooks resolve by project, and a stopped session isn't in the list of active ones. Without the name, the receiving side could only do nothing, silently.
4. A function named _deep_merge_meta did a shallow dict.update. Worse, the status write reset the session's end timestamp whenever the state changed. The confirmation write had to re-assert "stopped", or it would restart the retention clock and re-archive every database every 14 days.

The design ended with four open questions. The coordinating agent answered them by measurement on the live host within four minutes, and one answer (167 owned databases, 0 ownerless) shrank a planned drain script to a spec. The task closed at 14:07.

Was July 15 planning or verification? Mostly verification, I think: the value came from checking the task's premises against the code, and the plan was the artifact that made the findings visible. A lighter pass that only verifies assumptions, without a full groom, is something I haven't tested.

The day before, a re-groom of the same work had already paid once. Someone read the old reaper instead of trusting the spec's description of it, and found a live bug: it was dropping stopped sessions' databases after ten minutes, so any session resumed later got a fresh empty clone in production. Nobody had noticed.

Planning has a reach, and it ended there: the rollout still found three bugs that no green suite caught, which is a different article. The plan paid because of what it found in the code. The question is whether a plan on every ticket finds anything.

Two Studies of Coding Agent Planning on Small Tickets

The first study tested a separate planner. For every ticket, one session read the source and wrote a spec, and a fresh agent implemented it. I ran it on 24 clear public issues from Django, SymPy, Sphinx and Pylint, three times each. Success barely moved: 71 of 72 attempts passed without a plan and 68 of 72 in the planner condition, including seven runs where the planner delivered no spec. Two of those four failures were valid fixes the tests rejected. With tasks this close to the ceiling, I can say I saw no gain, not that there is none.

The time did move. Planning added 69% to the model time for the same tickets.

The second study asked a different question: does planning cut the slop that tests don't catch? I count two kinds. Reinvention is a change that rewrites a helper, pattern or test the codebase already had within reach. Name trust is a change that relies on a name or a local pattern when code elsewhere decides the real behaviour. A judge scores each run by the share of known chances for those two mistakes that it took. Zero is clean, and lower is better. It's a narrow measure, not general code quality: two mistake types, counted only where tests don't catch them, on the 10 tasks that had such chances. Think of a commit() that doesn't commit (more on that later).

Here the groom ran inside the implementing session, before any code, in two strengths. The context-only plan committed just the context the agent gathered. The full groom committed a plan in my own grooming template. I compared both with no plan on 13 small tasks from Django, Celery, CPython, Home Assistant and a codebase a team I work with maintains. The tickets came thin, rich, or rich with one planted false claim. Two models ran them, Claude Sonnet 5.5 and gpt-6.1-sol, for 264 judged runs. Each model's work was judged by the other model's family, so the study can't tell you which model slops less.

The three-sentence rule was mine to test. My worker grooms a ticket when it has under three sentences or no acceptance criteria, and the study's thin tickets were about that length. If the rule were right, the groom would help thin tickets more than rich ones.

It didn't show that. On thin tickets, the full groom moved slop by -0.025 on that zero-to-one scale. The 95% interval runs from -0.150 to +0.075, so the real effect could be a little better or a little worse. The extra help the rule predicts for thin tickets came out at zero. Leave out the private codebase and both numbers tip against the groom, to +0.050 and +0.100. Neither number can bear much weight.

So here's what I can conclude. On these small tasks, I didn't see grooming reduce slop. That isn't the same as showing it does nothing. In the main comparison, each combination of task, ticket, plan and model ran once, and results that sparse can't tell a small benefit from no change.

One result did stand out, in the wrong direction. A context-only plan on thin tickets produced slightly more slop than no plan. It was a secondary comparison, and two Celery tasks carry it, so I read it as a hint, not a finding.

Wrong Claims in Tickets, on Paper and in Code

Each poisoned ticket was the packaged task plus exactly one confident claim the base code refutes, the shape of "this call commits".

The no-plan baselines don't settle it either. Sonnet acted on the planted claim in 5 of 16 runs without a groom, sol in 2 of 16. Moving from 5 to 3 for one model and from 2 to 3 for the other is too thin to call a benefit. A planted claim also says nothing about how often real tickets carry one.

Real tickets do carry them. My own sketch on July 15 was a detailed, confident, agent-written ticket with four facts wrong. Another case, from a team I work with: a ticket from a colleague's AI-assisted audit called a public upload flow a vulnerability and asked for authentication on it. The groom half-corrected the framing, then made a scoped grant for anonymous callers a requirement, designed two token schemes for it and spread the grant into two sibling tasks. Nine days later, after a design review with me (I designed that API), a re-groom struck it before any code was written. The same audit produced two more tickets, and the grooms found claims the code contradicted in both. A groom can push back on a false claim or launder it into requirements.

A groom can push back on a false claim or launder it into requirements.

Where Planning Time Goes

In the second study, the agent spent a third to a half of the session, about 90-130 seconds, before it committed its plan. That part is observable: gross time that includes the exploration the plan needed. Every one of the 128 groomed runs committed its plan before writing production code.

How much time requiring a plan added is a different question, and the study doesn't isolate it. Comparing cells, the typical session moved by about a minute at most against the same ticket type without a plan, though Sonnet's averages with the full groom grew by one to two minutes. My guess, not something the study observed, is that the plan replaced some exploration the agent does anyway.

Money is the third measure. Sonnet's runs cost about the same with or without the groom, 40 to 50 cents each. The clear cost number is still the first study's: a separate planning session added 69% to model time. These in-session differences measure something else. Across these small tasks, the pre-plan phase took a third to a half of the session, with no detected drop in slop from the full groom.

My prior, stated plainly: I want grooming's money, token and wall-clock costs measured, and I don't expect those numbers to make me groom less.

Where I Put the Planning Instead

My unit of planning is the whole piece of work, not the ticket. I talk it through with an agent first, usually for one or two hours, and for the largest pieces over days of reworking a design document. That gives a skeleton: the decisions I've made, plus the knowledge base, the docs, existing code patterns, wide unit tests and CI. Then one groom fills in the detail against that skeleton, and every subtask inherits it instead of planning on its own. Parallel streams still need a sync, which costs more than people expect.

Here's how I route work. This is my practice, not a finding:

• A rich source goes straight out. A detailed ticket or a long conversation has already filled the gaps. I read it first, because agents write tickets too.
• A bare idea gets a groom. If the idea doesn't fix the shape I want, an unplanned worker will duplicate code or miss nuances.
• A whole piece of work gets one plan, as above. Its subtasks don't plan again.
• A subtask goes back to the shared plan when it hits a decision nobody made or a dependency the skeleton doesn't settle.

On Takt, my task board, the worker skill encodes a fallback for the second case. It grooms when the description is under three sentences or has no acceptance criteria, carries an incomplete-description label, or has a multi-phase title with no subtasks. It skips when the task already has context, requirements, notes and criteria, when it's a small bug with clear repro steps, or when it's a subtask whose parent carries the context. After a groom, a single task continues in the same session. The sentence count is my fallback, and the study didn't support it.

The second study didn't test any of this. What I have instead are records: 57 of my own pieces of work, mined from the board and session history. Every one had a planning step, so there's no unplanned control to compare against. Plans adapted in-line: 23 of 35 added tasks after work began, agents made 61 of the 63 later spec edits, and only 5 got a second groom.

Six of the eight I read closely still needed steering or a re-plan. In one, the design task ran alongside the code, and its review found three high-severity defects mid-build.

The two with the least steering had the most decisions written down before the groom.

A webhook-permissions feature started from six recorded owner decisions. A 16-minute groom split it into seven streams with explicit file ownership. All seven were done in about two hours, with two second-round reviews and one stranded fixup.

A token-model redesign started from a signed-off design and ran as eight serial phases in 18 hours. Review still found a high-severity defect on three consecutive phases. The design was wrong on one point, which had to be corrected in every later task brief, and smoke tests caught three more defects. Low steering, not zero rework.

That's an association in two cases, with human minutes unrecorded. The public version of the context step is agent-rails: its issue-worker skill reads the whole issue, writes out the acceptance criteria and must read docs before coding. A hook logs what it read, and the review step refuses when the log is empty.

Grooming Has Its Own Failure Modes

Given a small task, a groom sometimes designs a rocket engine. Without something to design against, it plants slop into the task itself. A task that implies persistence tempts a groom to invent a database layer the project never had.

My board shows the smaller versions. One groom split a task into three subtasks and cancelled them a minute later. A task sized S, two lines of a copy command, carried a 12,000-character spec.

The second study's plan audit agrees. Plans carried 0-0.7 unjustified scope items and 0.4-1.4 claims nobody had verified, per plan.

At a team I work with, a task permitted a new background subsystem even though an existing path would do, and the agent built exactly that. The tasks rewritten to port only what they named, with a stop-and-report tripwire, shipped clean.

That extends to the skeleton: agent-written docs, tests and specs inherit the same risk. My defence is decisions written down before the groom, which both low-steering cases had. Grooming fills in a skeleton. It doesn't replace one.

Some of that skeleton is knowledge only I had. While documenting a skill for scaffolding new projects, a session found its own project wasn't registered in Maestro and was about to fix that with maestro init. To the agent it sounded like the right way to initialise a project. It actually sets up a new machine to run sessions, and it would have overwritten the existing config with a half-broken one. The agents didn't know that, the groom didn't know that. Only I knew, because I'd been burnt by it before. A groom can only find what someone wrote down, so the skeleton's docs and checks have to hold it.

Judge a Plan by What It Found

I don't judge a plan by its length. The 22-row table is the model: every claim the ticket makes, checked against the code, with the result. These four things are also the minimum I ask a groom to hand back.

1. What it checked, and where. The files and docs it read for each claim. A checked fact can come back true, and a plan that confirms a constraint or narrows an unknown has done its job without contradicting anything.
2. What it will reuse, by name. The helpers, patterns and tests the change should build on, so reinvention is visible before the diff.
3. Which names it checked against their implementation. In a codebase I work in, a commit() call inside an open transaction doesn't commit. The fact lives deep in the database setup code, far from the call sites, and the code around those calls is full of commits that look real. We found it chasing deadlocks and wrote it into a design document. It was still re-learned three times, because the helper's own docstring said the opposite, until a test pinned it in August. Since June, about 1.4% of first code reviews there have flagged a wrong commit or transaction assumption.
4. Which assumptions remain open, and which findings changed the work. On July 15, four measurements answered the design's open questions, and one answer removed a script from the build.

For each factual claim in the ticket, I'd want one of five labels and a remedy. That's my advice, not a validated gate:

• Missing information: ask.
• A stale fact: reproduce or verify it.
• A false assertion: turn it into a question, not an instruction.
• A deliberate constraint: keep it, and mark it as a decision.
• A verified current fact: keep it, with the file or doc that confirms it.

The plan also needs permission to come back with "don't build this" or "already built". A plan that can only say yes is a delay. My slop checklist is for the diff. Reinvention and name trust are cheaper to catch here, in the plan.

A plan that can only say yes is a delay.

Plan Once, Then Read What It Found

Here's what I'd tell a friend. Don't put a planner in front of every ticket: its own failure mode is designing a rocket engine for a small job. Plan once, at the level where the decisions get made. Talk it through, give it a skeleton, and let one groom fill in what every subtask inherits. Before any code, verify the assumptions that could change the work.

Read the rich tickets, because some agent may have written them too. For thin ones, there's a line somewhere, but it sits lower than my three sentences, which was a fairly random spot. A ticket that's just a title means someone mines context before touching the code, and agents on modern models may do that as well as I would. My study's thin tickets were about three sentences, so the title-only case is untested. I'd plan where premises or decisions are unresolved, not where a description is merely short.

On small, clear tickets, my two studies didn't detect a gain from planning every one. The plan that paid on July 15 was a different unit. It earned its keep by retiring uncertainty (four wrong facts and four open questions), not by producing more specification.

Next time an agent hands you a plan, don't read it top to bottom. Ask what it checked in the code and the docs, what it will reuse, which assumptions are still open, and which findings changed the work.