Lesson 3: Drive a bounded change, test by test
Objective
The starter passes all nine tests through a sequence of bounded changes, and
you can explain every diff the agent produced without reading the solution
first.
Why this lesson exists
This is the lesson people skip — and the one that builds the actual skill.
Watching an agent make seven failing tests pass in one giant rewrite teaches
you nothing. Supervising seven small diffs, each pinned to a contract line,
teaches you how to keep AI-written code yours.
The lesson
Tell the agent: “Work through TASK.md against starter/report_tool.py. One
failing test group at a time: JSON loading, then validation isolation, then
rounding, then atomic writes, then the end-to-end tests. After each group,
run the suite and show me the diff before continuing.”
Hold it to the contract’s boundaries:
- Only
starter/report_tool.py changes. If a diff touches tests/,
solution/, or scenario/, stop and ask why.
- No new imports outside the standard library — and no import that is not
needed by the change being made.
- Each change should move toward one contract line. Reject drive-by refactors
(“while I was here I renamed…”).
- If the agent wants to change a test, the answer is no. Tests are the
contract; the code moves.
Run the suite after each group:
PYTHONPATH=starter python -m unittest discover -s tests -v
Expect the failing count to drop group by group: 7 → 5 → 4 → 3 → 2 → 0.
Exercise
Do the last group yourself. When only the end-to-end tests remain, write the
run_scenario/main fix by hand (they are small), then run the full suite.
Reading the agent’s diff is learning; writing the last ten lines is
internalizing.
Checkpoint
python verify.py starter --expect-failure # must now FAIL the expectation: starter passes
python verify.py starter # must pass all nine tests
The first command failing is good news — it means the starter is no longer
correctly unfinished. Then answer: which change was smallest? Which would you
have over-built?
Expected evidence
Test transcript from 7 failures to 0, plus the diffs. You are allowed to open
solution/ only after your starter passes — compare approaches, then note one
thing the reviewed solution does that yours does not.