AI Coding Live News

Teams from UC Berkeley, Princeton and others released FrontierCS 2 (arXiv 2610.29932): 48 tasks across 13 CS domains, each built by deleting a paper's contribution from the authors' released code. An agent has 5 hours to write it back and passes only if it beats the original on every metric and every hidden workload. Of 12 agents, Claude Opus 5.5 in Claude Code leads with a 58% pass rate, every other agent passes fewer than half, and 16 tasks have no passing run. On Oct 26 maintainer Wenhao Chai said a trace audit showed the Pi harness ends runs on refusals; rerun in Antigravity, Gemini 4 Argon now beats GPT-6 Astra and sits right behind Opus 5.5.

Agentic post-training (scaling multi-turn RL and SFT) requires massive GPU investments, but AI labs lack reliable heuristics to predict which base model checkpoint possesses the highest potential before committing to multi-million-dollar post-training runs. Cold-start zero-shot evaluations on SWE-bench fail because unaligned base checkpoints cannot emit formatted tool invocations, while single-turn patch benchmarks ignore multi-step state management. Researchers from NVIDIA and UC Berkeley (including Bryan Catanzaro and Ashish Vaswani) present a lookahead evaluation framework (arXiv:2610.10478). By replaying successful post-trained trajectories and rerunning unit tests after every edit, they isolate the 'decisive step' that flips a repository from failing to passing. They introduce three lightweight evaluation screens at this decisive step that require zero cold-start tool calling: Decisive-Action Bits Per Byte (BPB), Patch MCQ, and prefix-conditioned pass@K. Across ten pairs of public base and post-trained models, these screens rank candidate checkpoints in tight agreement with final post-trained SWE-bench Verified pass@1 scores, establishing a low-cost lookahead methodology for model selection.