FlyPython
English
Sign inStart a challenge
English中文

Hands-on with GitHub Copilot · Challenge 04

Lesson 4: Verify like a reviewer, not a spectator

Checkpoint l0410 ptsgated by test suiteGitHub Copilot in VS Code

Green tests are the floor, not the ceiling — run both suites, walk a scenario skin, and review the accumulated diff in Source Control the way you would review a colleague's.

Six agent-tool courses share this challenge. GitHub Copilot in VS Code-specific steps are still being written — until they land and pass review, this page defers its canonical to the primary course's copy.

Doing this with your agent? One sentence starts the whole course:

Read https://flypython.com/skills/flypython/SKILL.md and start the FlyPython course `hands-on-with-github-copilot`.

Lesson 4: Verify like a reviewer, not a spectator

Objective

Both suites pass, and you hold a written record of what the green runs do and do not prove — plus a reviewed diff you would defend to a colleague.

Why this lesson exists

“All tests pass” is where most AI-assisted work stops — and where most silent bugs start. A suite proves the contract lines it asserts; it says nothing about the rows nobody wrote a test for. Verification is a report you write, not a feeling you have.

The lesson

Run the full pair in the integrated terminal yourself — not through the agent — so you see the raw output:

python verify.py starter     # the implementation you supervised
python verify.py solution    # the reviewed reference

Both green means the contract is satisfied as written. Now do the review the suite cannot do for you:

  1. Walk a skin end-to-end. Pick a scenario/<skin>/ folder and run the tool on it: python starter/report_tool.py scenario/<skin>. Read the produced report against the input file — recompute one group total by hand.
  2. Review the accumulated diff. In Source Control, compare against the baseline commit from Lesson 3 (or run git diff <baseline> in the terminal). Read it top to bottom. Every hunk should map to a contract line you named in Lesson 2. Anything that does not is scope creep — revert or justify it. Also check for untracked files the agent created through terminal commands: those do not always appear in the chat’s changed-files list.
  3. Check the edges the tests skip. Feed the tool an empty CSV, a CSV with only invalid rows, a missing file. The contract does not pin these — your judgment does. Write down what it does.

If you want a second opinion, ask Copilot in plain words to critique the change (its built-in Rubber Duck reviewer is read-only). Treat whatever it finds as leads to check by running something, not as evidence.

Exercise

Write the three-line evidence record for this change:

  • Verified: the commands you ran and what they proved.
  • Not verified: the inputs and behaviors no test pins.
  • Known limits: anything you saw that the contract never promised.

Keep it honest — “not verified” is a list of facts, not a confession.

Checkpoint

Run python verify.py. This checkpoint’s code prints once the starter suite passes; the maintainer pair above must both be green for the gate to mean what it says. You pass the lesson when you can answer:

  1. What did your by-hand recompute of a group total show?
  2. Name one input the suite does not pin and what the tool does with it.
  3. Which hunk, if any, would you ask a colleague to justify?

Expected evidence

Both green runs, your three-line evidence record, and the diff review notes.

Hints

Stuck? Open one at a time.

Hint 1 — What this checkpoint tests

That starter AND solution both pass — the maintainer pair verify.py starter / verify.py solution must both be green, and python verify.py prints this code once your starter suite passes. The gate is the pair, not a single run.

Hint 2 — Review the delta, not the vibes

Source Control's diff against your baseline commit shows the whole accumulated change. Read it as a reviewer: every hunk maps to a contract line, nothing extra moved, no test weakened.

Hint 3 — When you are done

You have written three evidence lines — verified / not verified / known limits — and could defend the change in a code review.

Submit

Done? Record it.

Use the 8-character code printed by python verify.py. For a self-reported checkpoint, answer the lesson questions first, then run python verify.py --attest l04.

Youbrowser

Submit the claim code

Your agentauto-submit

Let the agent solve and submit

Hand it this checkpoint (copy button on hover) — it solves, verifies, and submits on its own:

Work on the FlyPython challenge "Hands-on with GitHub Copilot" (course id course-github-copilot), checkpoint l04.
Machine-readable brief: https://flypython.com/api/challenges/hands-on-with-github-copilot
Open the course folder and read TASK.md first — it is the contract.
Rules: smallest change, no new dependencies, never edit tests/ or solution/.
Check starter/ with python verify.py; ask me the lesson questions before using --attest ID for any self-reported checkpoint.
Target: solve only checkpoint l04, and submit it as soon as it passes.

With a token from your agent page (env FLYPYTHON_TOKEN), it records the result directly:

curl -X POST https://flypython.com/api/claims \
  -H "Authorization: Bearer $FLYPYTHON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"claims":[{"course":"hands-on-with-github-copilot","checkpoint":"l04","code":"<8-char code>"}]}'

Or install the full FlyPython agent skill once and skip the paste.

Files

The starter and verifier

Solving runs on your machine — your agent fetches the course files itself via /api/challenges/hands-on-with-github-copilot/files; you download nothing by hand. Prefer reading the source? Browse the folder on GitHub ↗