FlyPython
Sign inStart a challenge

Hands-on with the OpenAI Codex App · Challenge 04 · 中文版

Lesson 4: Verify like a reviewer, not a spectator

Checkpoint l0410 ptsgated by test suiteOpenAI Codex desktop app

Green tests are the floor, not the ceiling — run both suites, walk the scenario skins, and review the diff the way you would review a colleague's.

Six agent-tool courses share this challenge. OpenAI Codex desktop app-specific steps are still being written — until they land and pass review, this page defers its canonical to the primary course's copy.

Doing this with your agent? One sentence starts the whole course:

Read https://flypython.com/skills/flypython/SKILL.md and start the FlyPython course `hands-on-with-openai-codex`.

Lesson 4: Verify like a reviewer, not a spectator

Objective

Both suites pass, and you hold a written record of what the green runs do and do not prove — plus a reviewed diff you would defend to a colleague.

Why this lesson exists

“All tests pass” is where most AI-assisted work stops — and where most silent bugs start. A suite proves the contract lines it asserts; it says nothing about the rows nobody wrote a test for. Verification is a report you write, not a feeling you have. The Codex app’s thread view makes this concrete: the diffs, the commands, and their outputs are all on the same task — review them together.

The lesson

Run the full pair, in the app’s terminal or in the thread:

python verify.py starter     # the implementation you supervised
python verify.py solution    # the reviewed reference

Both green means the contract is satisfied as written. Now do the review the suite cannot do for you:

  1. Walk a skin end-to-end. Pick a scenario/<skin>/ folder and run the tool on it: python starter/report_tool.py scenario/<skin>. Read the produced report against the input file — recompute one group total by hand.
  2. Review the accumulated diff. In the thread’s diff view, read the whole change top to bottom. Every hunk should map to a contract line you named in Lesson 2. Anything that does not is scope creep — ask the thread to justify or revert it.
  3. Check the edges the tests skip. Feed the tool an empty CSV, a CSV with only invalid rows, a missing file. The contract does not pin these — your judgment does. Write down what it does.

Exercise

Write the three-line evidence record for this change:

  • Verified: the commands you ran and what they proved.
  • Not verified: the inputs and behaviors no test pins.
  • Known limits: anything you saw that the contract never promised.

Keep it honest — “not verified” is a list of facts, not a confession.

Checkpoint

Run python verify.py. This checkpoint’s code prints only when both suites pass. You pass the lesson when you can answer:

  1. What did your by-hand recompute of a group total show?
  2. Name one input the suite does not pin and what the tool does with it.
  3. Which hunk, if any, would you ask a colleague to justify?

Expected evidence

Both green runs, your three-line evidence record, and the diff review notes.

Hints

Stuck? Open one at a time.

Hint 1 — What this checkpoint tests

That starter AND solution both pass — verify.py prints this code only when both suites are green. The gate is the pair, not a single run.

Hint 2 — Review the delta, not the vibes

The app keeps the whole thread's diff reviewable. Read it as a reviewer: every hunk maps to a contract line, nothing extra moved, no test weakened.

Hint 3 — When you are done

You have written three evidence lines — verified / not verified / known limits — and could defend the change in a code review.

Submit

Done? Record it.

The 8-character code verify.py progress printed for this checkpoint — from the browser, or straight from your agent.

Youbrowser

Submit the claim code

Your agentauto-submit

Let the agent solve and submit

Hand it this checkpoint (copy button on hover) — it solves, verifies, and submits on its own:

Work on the FlyPython challenge "Hands-on with the OpenAI Codex App" (course id course-codex-cli), checkpoint l04.
Machine-readable brief: https://flypython.com/api/challenges/hands-on-with-openai-codex
Open the course folder and read TASK.md first — it is the contract.
Rules: smallest change, no new dependencies, never edit tests/ or solution/.
Check with python verify.py until its gates pass, then report each claim code to me.
Target: solve only checkpoint l04, and submit it as soon as it passes.

With a token from your agent page (env FLYPYTHON_TOKEN), it records the result directly:

curl -X POST https://flypython.com/api/claims \
  -H "Authorization: Bearer $FLYPYTHON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"claims":[{"course":"hands-on-with-openai-codex","checkpoint":"l04","code":"<8-char code>"}]}'

Or install the full FlyPython agent skill once and skip the paste.

Files

The starter and verifier

Solving runs on your machine — your agent fetches the course files itself via /api/challenges/hands-on-with-openai-codex/files; you download nothing by hand. Prefer reading the source? Browse the folder on GitHub ↗