AI Coding Live News

webAI CEO David Stout announced on X on November 4 (17:37 UTC) that the company is open-sourcing TwIL-8B. The official Hugging Face model card says it is post-trained from IBM Granite 4.2 8B with full-parameter SFT, GRPO/DAPO reinforcement learning and a length-aware RL stage, and ships in balanced and verbose variants. Self-reported scores include LiveCodeBench 81.6 (verbose), MATH-500 98.2 and IFBench 74.4, about 93% and 92% of the best frontier score, while GPQA Diamond is only 70.2. A 4.98 GiB Q4_K_M GGUF runs locally; the license is the webAI Non-Commercial License.

Zoom

Researchers from Amazon Web Services (AWS) and UIUC unveiled Lean4AgenticRL (arXiv:2610.22448). Tackling the limitations of conventional agentic reinforcement learning relying on sparse, binary outcome rewards, the framework grounds multi-level process rewards within a formal Lean4 verification schema. Utilizing workflow-stage-normalized GRPO, the resulting LAR-27B model reaches 77.3% on TerminalBench-2.1, outperforming base models by 9.89% and surpassing comparable open models by 5.07%.

Zoom

Deep search agents struggle with credit assignment across long, noisy retrieved contexts. Zhejiang University and Tencent open-sourced KnowGPO (arXiv:2610.22331, GitHub: ZJU-REAL/KnowGPO), an RL framework that groups search trajectories by atomic knowledge conditions rather than surface observations. Qwen3-8B fine-tuned with KnowGPO achieves 38.8% on BrowseComp-ZH and 40.2% on GAIA, outperforming GRPO/GiGPO baselines and rivaling 32B open-source search systems.

Zoom

To combat goal drift in long-horizon agent tool-use and multi-agent systems, NUS, USTC, and Alibaba Qwen introduced Steering Agent (arXiv:2610.22316). Jointly trained via pairwise ranking and Group-Relative Policy Optimization (GRPO), the agent operates a low-cost fast path to detect drift and a slow path to generate grounded interventions. It raises Multi-Hop QA benchmark performance from 42.70 to 50.10 while cutting executor turns from 8.8 to 6.5 (-26.1%) without modifying the underlying executor models.

Zoom

Three new studies tackle verification, evaluation, and RL alignment for rule-governed generative world models: OmniVerifier-C1 (arXiv:2610.22355, 32 upvotes) uses Decoupled RL to train a decomposed graded visual verifier that reaches 0.684 on ViVerBench and boosts FLUX.1-dev GenEval from 0.68 to 0.79 while enabling Maze and Sudoku world modeling; Meituan's WorldCast benchmark (arXiv:2610.20731, 30 upvotes) evaluates 12 models across 320 multi-step world state prediction cases, finding GPT-Image-2.5 System at 72.8 vs. 42.8 for Nano Banana Pro and 20.1 for the top open model; and NUS's Routing Policy Optimization (RPO, arXiv:2610.22282, 23 upvotes) turns attention routing into an internal exploration action to train video/image diffusion RL 3.1x–4.2x faster than Flow-GRPO and DanceGRPO.

Zoom

Addressing rollout homogenization and zero-variance advantage collapse in group-relative reinforcement learning (GRPO), SERO (arXiv:2610.19397) restores learner-visited states to execute multi-step oracle continuations with policy-reachability weighting—boosting 7B ALFWorld OOD success by +11.20 points to 87.50%—while UNSW's EDEN (arXiv:2610.16105) appends a single anchored Gaussian-perturbed embedding position to a fraction of rollouts, lifting Qwen2.5-Math-7B AIME 2025 Pass@256 by +13.33 points (to 66.67%) without increasing rollout budgets.

Zoom

Off-policy data in reinforcement learning with verifiable rewards (RLVR) causes severe policy gradient bias during asynchronous generation and response reuse. Researchers from UIUC and HKUST introduce Proximal KL, theoretically proving that full future KL credit renders the exact proximal optimizer the unique stationary policy without requiring full-response weights or fitted critics. Evaluated across Qwen3-1.7B and 4B models, Proximal KL outperforms GRPO on 8 math benchmarks by up to 2.58% while completely averting training collapse.

Zoom

Controlling acting LLM agents requires step-level confidence signals to decide whether to execute an action, abstain, or hand off control to humans. Research from NYU introduces outcome-grounded confidence calibration for multi-step agents. Rather than calibrating confidence against immediate local actions, the framework aligns per-step verbalized confidence with the ultimate multi-step episode outcome using GRPO rollouts and proper scoring rules. Evaluated across ALFWorld, TextWorld, and WebShop on Qwen2.5-7B, confidence AUROC jumps to 0.68–0.85, steps within the top-10% confidence bracket succeed 4.3x more frequently, and expected calibration error plummets from 0.42 to below 0.12.

Zoom

To improve credit assignment in long-horizon agentic tasks, combining reinforcement learning with verifiable rewards (RLVR) and on-policy self-distillation (OPSD) has emerged as a leading paradigm. However, existing pipelines neglect the dynamic evolution of agentic skills and the reliability of privileged teacher signals. Researchers from CUHK and Tencent introduced RSIPO, enabling skills and policies to co-evolve recursively. RSIPO extracts skills directly from trajectory rollouts, optimizes skill generation via replay, and employs Empirical Bayes shrinkage to calibrate student trust in teacher logits. Operating on Qwen2.5-7B, RSIPO achieves 98.6% success on ALFWorld, outperforming standard GRPO by 17.4 percentage points.

Zoom

Salesforce AI Research has introduced Prospective Hindsight (PH), a self-calibrating training framework for long-horizon autonomous agents. Addressing the fundamental limitation of standard RL where credit assignment occurs strictly after environmental feedback, PH exploits the gap between the agent's pre-action prospective prediction and post-action retrospective evaluation. By weighting gradients with this surprise residual, PH eliminates overconfident failures in single-turn tasks and underconfident successes in multi-turn interactions across GRPO and on-policy distillation.

Zoom

Robotic manipulation and physical intelligence systems suffer when relying on general-purpose vision backbones that struggle with dense clutter, tiny objects, and closed-loop control speeds. GroundingPI establishes a dedicated 4B grounding foundation model that unifies points and bounding boxes as quantized coordinates within a shared vocabulary. Outperforming 44 baselines across 34 benchmarks with an average of 73.68% (surpassing GPT-6 Astra at 71.54%), GroundingPI sets a new standard for embodied robotic visual backbones.

Zoom

Reinforcement Learning with Verifiable Rewards (RLVR) powers modern reasoning LLMs by translating post-interaction scalar outcomes into optimization signals. However, in group-relative objectives (e.g., GRPO), reward signals vanish completely when all rollouts within a group share the same score (such as widespread mutual failures), discarding rich diagnostic feedback regarding task requirements and failure modes. In smaller models (such as 2B parameters), up to 98% of rollout groups result in universal failure, leaving standard RLVR stranded at 0% task success. Salesforce AI Research introduces Self-Retrospection Distillation (SRD, arXiv:2610.08077, repo: github.com/SalesforceAIResearch/SRD). SRD instantiates prospective learning, using privileged post-hoc trajectories to supervise pre-interaction foresight representations within the same policy without adding inference latency. Across 10 tool-integrated reasoning and long-horizon agentic benchmarks, SRD delivers gains of up to 24.2 percentage points over baseline RLVR and self-distillation. Critically, in 2B model regimes where 98% of groups are all-failure and standard RLVR flatlines at 0.0%, adding SRD reaches a 60.6% success rate under identical rollout budgets.

Zoom