Zero-Human-Label Step Accuracy Jumps from 48.2% to 74.7%: LeanPRM Grounds Process Supervision in Lean 4, While Cornell's CPLM Matches Full Attention at w=128

Core Background & Industry Pain Points

Modern LLMs face two fundamental bottlenecks in mathematical proof verification and long-context retrieval: First, as models generate multi-page natural-language mathematical proofs, verifying step-by-step validity becomes the primary bottleneck—open-ended proofs lack scalar final answers for outcome-based RLVR, human step annotation (PRM800K) is prohibitively expensive, and manually formalizing every proof in Lean requires expert labor. Second, while Transformers are widely assumed to excel at copying tokens from context, theoretical and empirical analysis from Cornell University reveals that the standard Language Modeling (LM) head suffers from a gradient bottleneck when jointly optimizing semantic generation and exact in-context copying.

Architectural Highlights & Core Mechanisms

Two new papers—LeanPRM (arXiv:2610.17697) and Copy-Paste Language Models (CPLMs) (arXiv:2610.19247)—resolve these challenges through formal-kernel process supervision and explicit copy-head modeling:

• LeanPRM: Lean-Grounded Step Supervision Without Human Labels: A generator LLM produces a natural-language proof paired step-by-step with a formal Lean 4 proof without interactive Lean feedback. The Lean compiler checks every step, and a secondary filter discards failures caused solely by formal syntax so that negative labels pinpoint genuine mathematical errors. This automated pipeline yields 10,634 proofs with 152,000 step-level labels and zero human annotation.
• Copy-Paste Language Models (CPLMs): Augments the standard LM head with a lightweight Copy-Paste (CP) head (~3% compute overhead) that creates an alternative gradient path bypassing the vocabulary projection bottleneck. When paired with sliding-window attention (SWA), the CP head enables direct token retrieval across the full context without requiring expensive interleaved full-attention layers.

Authoritative Benchmarks & Empirical Comparisons

• LeanPRM Beats PRM800K by +14.4 Points (arXiv:2610.17697): Training a Qwen3-4B verifier on LeanPRM boosts exact step-label accuracy on the 200-proof held-out LeanPRM-Bench from 48.2% to 74.7%, compared to 60.3% when the same base model is trained on human-annotated PRM800K. Across model scales from 0.6B to 8B, LeanPRM improves performance on LeanPRM-Bench, Hard2Verify, IMO-GradingBench, and Best-of-$N$ proof selection on IMO-level problems.
• CPLMs Across 135M–1.7B Scales & 4x Context Extrapolation (arXiv:2610.19247): Pretraining models from 135M to 1.7B parameters, CPLMs outperform vanilla baselines across zero-shot benchmarks by +0.4 to +0.9 points overall and +2.5 to +4.0 points on LAMBADA, while lowering perplexity by 0.5–2 points on FineWeb-Edu and up to 15 points on OOD FinePDFs/arXiv. Remarkably, a sliding-window CPLM with a tiny window of $w \simeq 128$ matches the perplexity of a full-attention vanilla Transformer while using far less memory and continuing to improve on 16K-token documents (4x pretraining context).

Developer Practical Guide & Quickstart

Reasoning teams can explore the LeanPRM dataset, models, and LeanPRM-Bench evaluation suite at https://rlafuente.com/lean-prm to plug automated step-level verifiers into Best-of-$N$ or RL post-training pipelines. Pretraining engineers designing memory-efficient sliding-window architectures can attach CPLM's lightweight copy-paste head to retain full-context token copying fidelity at ~3% compute overhead.