Block merge on unchecked test-plan checkboxes in the PR body

[justin808]

Problem

PR bodies ship with unchecked test-plan checkboxes and merge anyway. The checklist is written by the authoring agent, never executed, and nothing notices.

Worked example: react_on_rails PR #2489 (d0d193a0d, merged 2026-03-05) merged with this in the body:

## Test plan

- [ ] Verify the generator produces correct `serverWebpackConfig.js` with `--rspack` flag
- [ ] Verify existing webpack (non-rspack) behavior is unchanged
- [ ] Review migration docs render correctly on the documentation site

All three unchecked. Item 1 was in fact never done in any form: four months later there is still no test asserting the rspack behavior that PR fixed.

This is the cheapest possible signal we are currently throwing away. The agent wrote down what it should verify, then did not, and the artifact recording that is sitting in the PR body at merge time.

Proposal

Block merge on an unchecked - [ ] in the PR body.

Mechanics:

• Scope the check to the PR description, not comments.
• Default to scanning the whole body. A narrower variant (only under headings matching Test plan, Verification, Checklist) is more precise but more fragile to heading drift; whole-body is simpler and the false-positive cost is one keystroke.
• Resolution paths for the author: check the box after doing it, strike it through or delete it if it does not apply, or move it to a follow-up issue and link that issue.
• Wire it into the same merge-precondition path as Follow-up: Fail closed on unsettled or failed configured reviews before merge #249 so both surface in one receipt, and make it overridable at the repo seam only with an explicit waiver that names the unchecked items.

Why a lint and not prose

The pack already tells agents to verify their work. Prose did not stop this. This one is machine-checkable in a few lines against gh pr view --json body, has no false-negative ambiguity, and fails closed.

A useful side effect: it makes agents stop writing aspirational checklists they do not intend to execute. Either the box gets checked or the line gets deleted, and both outcomes are more honest than what we have now.

Related

• Follow-up: Fail closed on unsettled or failed configured reviews before merge #249 (fail closed on unsettled or failed configured reviews before merge). Same merge-precondition surface.
• The coverage-of-change gate proposed separately covers the harder version of this problem. This issue covers the trivial version that requires no new judgment at all.