FlyPython
Sign inStart a challenge

Hands-on with DeepSeek Harness · Challenge 04 · 中文版

Lesson 4: Verify like a reviewer, not a spectator

Checkpoint l0410 ptsgated by test suiteDeepSeek Harness

Green tests are the floor, not the ceiling — run both suites, walk the scenario skins, and review the trajectory the way you would review a colleague's work.

Six agent-tool courses share this challenge. DeepSeek Harness-specific steps are still being written — until they land and pass review, this page defers its canonical to the primary course's copy.

Doing this with your agent? One sentence starts the whole course:

Read https://flypython.com/skills/flypython/SKILL.md and start the FlyPython course `hands-on-with-deepseek-harness`.

Lesson 4: Verify like a reviewer, not a spectator

Objective

Both suites pass, and you hold a written record of what the green runs do and do not prove — plus a trajectory you have reviewed end to end.

Why this lesson exists

“All tests pass” is where most AI-assisted work stops — and where most silent bugs start. A suite proves the contract lines it asserts; it says nothing about the rows nobody wrote a test for. Verification is a report you write, not a feeling you have. The harness’s append-only trajectory makes the review concrete: every command and edit is already logged — your job is to actually read it.

The lesson

Run the full pair:

python verify.py starter     # the implementation you supervised
python verify.py solution    # the reviewed reference

Both green means the contract is satisfied as written. Now do the review the suite cannot do for you:

  1. Walk a skin end-to-end. Pick a scenario/<skin>/ folder and run the tool on it: python starter/report_tool.py scenario/<skin>. Read the produced report against the input file — recompute one group total by hand.
  2. Review the trajectory end to end. Read every step the session took: the reads, the edits, the commands. Every edit should map to a contract line you named in Lesson 2; every verify run should actually appear. Anything unexplained is scope creep — replay or fork the session around it.
  3. Check the edges the tests skip. Feed the tool an empty CSV, a CSV with only invalid rows, a missing file. The contract does not pin these — your judgment does. Write down what it does.

Exercise

Write the three-line evidence record for this change:

  • Verified: the commands that ran (with their trajectory entries) and what they proved.
  • Not verified: the inputs and behaviors no test pins.
  • Known limits: anything you saw that the contract never promised.

Keep it honest — “not verified” is a list of facts, not a confession.

Checkpoint

Run python verify.py. This checkpoint’s code prints only when both suites pass. You pass the lesson when you can answer:

  1. What did your by-hand recompute of a group total show?
  2. Name one input the suite does not pin and what the tool does with it.
  3. Which trajectory step, if any, would you ask a colleague to justify?

Expected evidence

Both green runs, your three-line evidence record, and the reviewed trajectory.

Hints

Stuck? Open one at a time.

Hint 1 — What this checkpoint tests

That starter AND solution both pass — verify.py prints this code only when both suites are green. The gate is the pair, not a single run.

Hint 2 — Review the record, not the narration

The append-only trajectory is the auditable record of every command and edit. Review it end to end: every edit maps to a contract line, every command really ran, no test was weakened.

Hint 3 — When you are done

You have written three evidence lines — verified / not verified / known limits — and could defend the change in a code review.

Submit

Done? Record it.

The 8-character code verify.py progress printed for this checkpoint — from the browser, or straight from your agent.

Youbrowser

Submit the claim code

Your agentauto-submit

Let the agent solve and submit

Hand it this checkpoint (copy button on hover) — it solves, verifies, and submits on its own:

Work on the FlyPython challenge "Hands-on with DeepSeek Harness" (course id course-deepseek-harness), checkpoint l04.
Machine-readable brief: https://flypython.com/api/challenges/hands-on-with-deepseek-harness
Open the course folder and read TASK.md first — it is the contract.
Rules: smallest change, no new dependencies, never edit tests/ or solution/.
Check with python verify.py until its gates pass, then report each claim code to me.
Target: solve only checkpoint l04, and submit it as soon as it passes.

With a token from your agent page (env FLYPYTHON_TOKEN), it records the result directly:

curl -X POST https://flypython.com/api/claims \
  -H "Authorization: Bearer $FLYPYTHON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"claims":[{"course":"hands-on-with-deepseek-harness","checkpoint":"l04","code":"<8-char code>"}]}'

Or install the full FlyPython agent skill once and skip the paste.

Files

The starter and verifier

Solving runs on your machine — your agent fetches the course files itself via /api/challenges/hands-on-with-deepseek-harness/files; you download nothing by hand. Prefer reading the source? Browse the folder on GitHub ↗