Zero-Human-Label Step Accuracy Jumps from 48.2% to 74.7%: LeanPRM Grounds Process Supervision in Lean 4, While Cornell's CPLM Matches Full Attention at w=128
Tackling step-level verification in mathematical reasoning and in-context copying bottlenecks in Transformers, NYU/UCLA/Stanford researchers introduce LeanPRM (arXiv:2610.17697)—using the Lean 4 theorem prover to automatically label 152K steps across 10,634 natural-language proofs with zero human annotation, boosting Qwen3-4B step accuracy on LeanPRM-Bench from 48.2% to 74.7% (vs. 60.3% for PRM800K). Meanwhile, Cornell University's Copy-Paste Language Models (CPLMs, arXiv:2610.19247) add a lightweight Copy-Paste head (~3% compute overhead) that relieves the LM head gradient bottleneck, boosting LAMBADA accuracy by +2.5-4.0 points and allowing a w=128 sliding-window CPLM to match full-attention perplexity.
- •LeanPRM uses the Lean 4 proof assistant and a syntax-error filter to generate 152K step-level labels across 10,634 proofs with zero human annotation
- •A Qwen3-4B verifier trained on LeanPRM reaches 74.7% step accuracy on LeanPRM-Bench (vs. 48.2% base and 60.3% with PRM800K) and improves 0.6B-8B verifiers on Hard2Verify and IMO-GradingBench
- •Cornell's Copy-Paste Language Models (CPLMs) add a ~3%-overhead copy head that relieves the LM head gradient bottleneck, boosting LAMBADA by +2.5-4.0 points and matching full-attention perplexity with a w=128 sliding window






