Challenge course · Any coding agent + pytest

pytest in the Loop — Tests That Catch Bugs

A challenge course — given a correct library and a seeded buggy variant, write a pytest suite (fixtures, parametrize, raises) that passes on the reference and fails on the bugs. A suite that cannot catch a bug is not a suite.

Intermediate5 challenges1–2 hoursFoundation path graduatePro (subscription)Course badge: Suite That CatchesReviewed 2026-09-12English + 中文Pro

Start it with your agent — paste this one sentence:

Read https://flypython.com/skills/flypython/SKILL.md and start the FlyPython course `dt-pytest`.

Needs a coding agent that can run commands and reach the network (chat-only web AIs cannot). Your agent authorizes you with a one-time link, fetches this course's files itself, and you never download anything. Haven't installed the FlyPython Skill? Pick your agent's tool course →

Browse the challenges 中文版 →

What you learnA pytest suite that passes clean on the correct implementation and reports at least four failures on the seeded-bug variant — with fixtures, parametrized cases, and exception assertions.

Who it's forFoundation path graduate · Python 3.11 or newer available on your PATH

Duration & cost1–2 hours · Pro subscription

Challenges

Five lessons, each with a checkpoint

Every lesson ends with a command to run and questions you must answer without looking.

01

A suite that catches nothing is not a suite

Run the contract on the starter's weak suite and read why two green tests prove nothing.

Checkpoint + expected evidence

02

Define what the suite must prove

List the behaviors worth pinning — before letting the agent generate test code.

Checkpoint + expected evidence

03

Write the real suite

Replace the starter suite: fixtures, parametrized cases, exception assertions — until the contract passes.

Checkpoint + expected evidence

04

Prove it catches the bugs

The contract runs your suite against the buggy variant — fewer than four caught bugs is a fail.

Checkpoint + expected evidence

05

Test something you own

Port the loop to a function you own: one fixture, one parametrized edge case, one caught bug or an honest note.

Checkpoint + expected evidence

Start

Pro course — an active subscription unlocks it

ProPython developer toolchain with agents

Part of the Python developer toolchain with agents track

This course's files reach your agent through the same entry sentence above — with an active subscription and this track unlocked, the authorization link grants the pro:files scope automatically. Claims record only while the track is unlocked.

See pricing →Track overview →

Free forever

Free courses stay free

The foundation path and every community course in the open repository cost nothing — Pro funds the paid paths, it does not paywall the commons. All six agent-tool courses are free.

Start the free path →

Scenario skins

Same skills, a domain you recognize

All three skins run through the same code and the same tests. Pick the one closest to your job before lesson 1.

scenario/textstats/textstats.py

Text statistics library

A small stdlib library — word counts, reading time, slugify — with a deliberately buggy variant in bugs/.

Verification

How “done” is decided

Seven-test contract suite: your pytest suite must pass on the reference implementation, report ≥4 failures on the buggy variant, and contain a fixture, parametrized cases, and exception assertions — no skips allowed.

python verify.py  # from the course folder: checkpoint status + claim codes

Apply this to your own project

Lesson 5 ports the workflow — not the code — to one script you actually own: a three-line task contract, one new failing test made to pass, and a written record of what the tests do not prove. The same loop is described in the AI coding workflow guide and practiced by therunnable examples.

Claim your evidence

When a checkpoint passes, python verify.py prints a deterministic claim code. Your authorized agent submits them for you in one batch; by hand, paste them on yourprogress page — worth 10 points per checkpoint and theSuite That Catches course badge at five. Claim codes are self-reported evidence, not certificates.

What verification does not prove

Passing the course suite proves the pinned behaviors on the tested inputs — not correctness on tomorrow’s data, not production performance, and not anything about your own project until you write its contract. Repository tests are evidence, not your outcome.

Report teaching drift ↗All courses →