FlyPython
Sign inStart a challenge

From “it runs” to “it ships”: verifying AI-generated code · Challenge 04 · 中文版

Read the record like a skeptic

Checkpoint l0410 ptsgated by test suiteAny coding agent (Claude Code 2.x / Codex CLI 0.x / Cursor)

Green, red, and no-tests mean different things — prove you can tell them apart and say what the record does not prove.

Doing this with your agent? One sentence starts the whole course:

Read https://flypython.com/skills/flypython/SKILL.md and start the FlyPython course `verifying-ai-generated-code`.

Read the record like a skeptic

Objective

Green, red, and no-tests mean different things — prove you can tell them apart and say what the record does not prove.

The lesson

Run the finished check on all three scenario projects. Read each SHIP-RECORD.json: the same command, three different verdicts. The unverified list is not a disclaimer — it is the scope of your claim. Review the diff for unforced changes; try --verified-on and confirm the record is reproducible.

Exercise

  • Hand-check: does all_passed match exit_code and result in every record?
  • Write the three-line evidence note: verified / not verified / known limits

Checkpoint

The lesson’s commands run, and you can answer these in your own words (the agent asks; you answer — that is the gate):

  1. Which command did this lesson run, and what did it decide?
  2. What failed at the start, and why — in your own words?
  3. Before trusting a similar change next time, what would you check first?

Expected evidence

The command transcript and your answers.

Hints

Stuck? Open one at a time.

Hint 1 — What this checkpoint tests

Both suites plus reading records like a skeptic: green, red, and no-tests produce different verdicts — prove you can tell them apart.

Hint 2 — Run all three skins

run_project on green-project, red-project, and no-tests gives ok, failed, and no-tests — check each SHIP-RECORD.json against the contract, not against your memory.

Hint 3 — What the record cannot prove

State one thing a green record does not cover (like whether the tests themselves are any good) — that sentence is the lesson.

Submit

Done? Record it.

The 8-character code verify.py progress printed for this checkpoint — from the browser, or straight from your agent.

Youbrowser

Submit the claim code

Your agentauto-submit

Let the agent solve and submit

Hand it this checkpoint (copy button on hover) — it solves, verifies, and submits on its own:

Work on the FlyPython challenge "From “it runs” to “it ships”: verifying AI-generated code" (course id course-verify-ship), checkpoint l04.
Machine-readable brief: https://flypython.com/api/challenges/verifying-ai-generated-code
Open the course folder and read TASK.md first — it is the contract.
Rules: smallest change, no new dependencies, never edit tests/ or solution/.
Check with python verify.py until its gates pass, then report each claim code to me.
Target: solve only checkpoint l04, and submit it as soon as it passes.

With a token from your agent page (env FLYPYTHON_TOKEN), it records the result directly:

curl -X POST https://flypython.com/api/claims \
  -H "Authorization: Bearer $FLYPYTHON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"claims":[{"course":"verifying-ai-generated-code","checkpoint":"l04","code":"<8-char code>"}]}'

Or install the full FlyPython agent skill once and skip the paste.

Files

The starter and verifier

Solving runs on your machine — your agent fetches the course files itself via /api/challenges/verifying-ai-generated-code/files; you download nothing by hand. Prefer reading the source? Browse the folder on GitHub ↗