3.3.3Verify
Draft. Written, but not yet complete or polished.
Your agent says the feature works. Does it?
Without verification, you find out by checking yourself, which doesn't scale past one agent. That's why verification is the first bottleneck.
When the agent can check its own work, it can loop for much longer without your input: run a check, read the result, fix what failed, repeat. It comes back to you only when the work passes, not every time something breaks.
Verify and Validate both check whether the implementation is correct. The difference is who checks. In Verify, the implementing agent checks itself. In Validate, something other than that agent checks it, such as you or a fresh reviewer agent.
The rest of this page offers guidelines for verifying well. Most come from Addy Osmani's Agentic verification and Lauren Tan's The Complete Guide to pstack, Part 1.
Use end-to-end tests
End-to-end tests drive the app the way a user does, so they're hard to fool. When an agent writes code and unit tests together, both can share the same misunderstanding and pass anyway.
Keep a small set covering the flows that matter, since these tests are slow and flaky. Make failures diagnostic, with screenshots, traces, and the failing step.
Use property-based tests
An agent tends to pick test examples that match its own logic. Property-based tests instead state a rule that must always hold, and a tool such as fast-check (TypeScript), Hypothesis (Python), proptest (Rust), or Rapid (Go) generates thousands of inputs to break it, then shrinks any failure to its smallest case.
Keep the loop fast and deterministic
A flaky test teaches the agent to rerun instead of fix. A slow one makes it batch changes, so it can't tell which one broke things. Aim for a core loop of about a second, seed randomness and freeze clocks, and quarantine flaky tests on sight. Save end-to-end tests for before merge.
Let the agent drive your app
Tan argues the agent should be able to run, use, and debug your app as you would, for example through the Chrome DevTools Protocol. So choose a stack that allows it, and give the agent:
- A verification skill that says how to launch, drive, and observe the app, such as one written by create-verification-skill
- A feature map of what your app does and how to reach it, refreshed daily
- An agent-friendly CLI, with JSON output, rich
--help,--dry-runon destructive commands, and errors that say what to do instead
Check the checks
An agent can write tests that pass and prove nothing. Have it ask of each test: would this fail if the code were plausibly wrong? A check that needs someone without the builder's assumptions belongs in Validate.