Before / after
Before — original verdict
WRONG
agent_version
v0.1.0-sbert-stsb-2slice
After — current verdict
REPRODUCED
The reproduction was re-run under a corrected protocol and now sits at REPRODUCED. Click through to the paper page for the live evidence + claim citation.
Why the original verdict was incorrect
The driver evaluated the checkpoint sentence-transformers/bert-base-nli-stsb-mean-tokens against Table 1's unsupervised SBERT-NLI-large row (Spearman 79.23). The matching baseline is Table 2's supervised-base STS-b row (85.35). Once the comparison is repointed to the right row, the model lands within 0.5 points of the paper's number.
Evidence trail
- Audit thread — long-form post-mortem covering all seven 2026-05-13 retractions, including this one.
- Rollback PR — the GitHub pull request that landed the corrected driver and flipped the verdict row to
is_current=false. - All public retractions — the append-only retraction log under PRD §17.X.8(d).
- Verdict Validator — the C1/C2 gates that prevent this class of mistake from shipping again.
What changed structurally
The 2026-05-13 retraction rollup landed two structural fixes so the citation-side failure that caused the original incorrect verdict cannot ship the same way again:
- Typed claim citation per verdict. Every reproduction driver now declares a structured
CLAIM_CITATION(Table, row, column, reported value, quoted text, PDF page) before its Modal job runs. The original verdict on this paper was published against a non-citable headline — that path is now closed by the build-failing validator-wiring lint. - PDF-verified textual gate. The Verdict Validator fetches the cited paper's PDF and checks that the quoted text appears within ±200 characters of the cited reported value. Made-up, mis-cited, or category-confused citations fail this gate and the verdict is auto-downgraded.