Passing tests can lie.
reprove shows you when.
reprove verifies a pull request with evidence, not opinions. It runs your tests, measures coverage of the lines the change touched, mutates those lines to see whether the tests notice, and reports every finding as reproduced, inferred, or needs-evidence. Built for AI-generated and fast-built code that nobody has checked yet.
pip install reprove
reprove verify --base origin/main --no-llmThe case it exists for
A one-line change to a shipping threshold. A passing test. 100% line coverage of the change. And the tests cannot tell > from >= at the boundary the change is about.
reprove: release-safe evidence: weak
- 3 of 3 mutants on changed lines survived the tests
1. [high] Surviving mutant: tests do not detect this change - pkg/__init__.py:2 - status: reproduced
- checked: mutated total >= 100 -> (total > 100); ran tests/test_ship.py::test_weak; all passed
2. [high] Surviving mutant: tests do not detect this change - pkg/__init__.py:2 - status: reproduced
- checked: mutated 100 -> (101); ran tests/test_ship.py::test_weak; all passed
Coverage: changed lines executed by tests: 1 of 1
Mutants: killed 0, survived 3Write the boundary test and run it again:
reprove: release-safe evidence: strong
Coverage: changed lines executed by tests: 1 of 1
Mutants: killed 3, survived 0Same coverage number. Different evidence. Full demo, including what reprove says when the tests cannot run at all.
What it does
| Stage | What you get |
|---|---|
| Tests | Your project's own test command, run, with counts and the exact command. |
| Diff coverage | Which changed lines the tests execute, and which they never touch. |
| Mutation | Each changed line altered one operator at a time; the covering tests re-run; killed or survived, with the exact mutation. |
| Hypotheses (optional) | With your own model key: specific risks, each with a reproduction that reprove executes before showing it. |
| Report | Markdown for a PR comment and JSON. Verdict, findings by severity with file, line, command, output and status. |
What it is for, and what it is not
reprove answers did the tests notice this diff. It does not answer is this diff safe. The second question still needs a person with judgement. reprove does the reviewer's first hours, establishing what the tests actually exercise and whether they would fail, and hands over evidence. If you are hiring an independent reviewer for vibe-coded or AI-assisted code before launch, this is what you would want them to run first.
Offline by default. Tests, coverage and mutation never leave your machine or your CI. The only stage that sends anything anywhere is the optional hypothesis step, under your own key, only when you set it. If your code cannot go to a third-party model, do not set the key; everything else still runs.
In your pull requests
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@v4
with: { ref: ${{ github.event.pull_request.head.sha }}, fetch-depth: 0 }
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install -e ".[test]"
- uses: ping-dev-ui/reprove@v0.1.1One sticky comment per pull request, updated on every push.
Pricing
Private repositories: reprove Pro
Same code, same licence. Honour-based in this release. Pays for maintenance, gets email support, and puts your repository first in line for the roadmap.
SubscribeA few hours of a human reviewer's time costs more than a year of Pro.
Roadmap
In the order buyers of independent reviews ask for them: authorization and tenant-isolation checks; payment and webhook reproductions (duplicate delivery, retry after timeout, idempotency); dependency and licence audit on changed manifests; mutation for JS/TS; a one-word decline verdict when the evidence says do not release.
Findings with reproduced status will always be executed checks with logs. That rule does not change as the list grows.