GAGAR Reinvents Code Agent RL: Groupwise Agentic Grading and Advantage Redistribution Beyond Binary Test Rewards
Reinforcement learning (RL) pipelines for code agents typically rely on binary test suite outcomes for reward assignment. Under standard Group Relative Policy Optimization (GRPO), all test-passing rollouts within a sampled group receive identical advantage values, completely ignoring code quality, extraneous diff modifications, and algorithmic elegance. This induces severe trajectory bloat and training instability. Researchers introduce GAGAR (Groupwise Agentic Grading and Advantage Redistribution, arXiv:2609.32577). In a shared workspace, an agentic evaluator jointly inspects all passing candidates in a group, ranking them by architectural clarity and minimal intervention. A sum-preserving redistribution reallocates advantage weights toward superior implementations. Validated at industrial scale on MiMo-V2.6-Flash (310B) and MiMo-V2.6-Pro (1.02T parameters), GAGAR curtails trajectory length explosion while boosting code agent benchmark performance and optimization stability.
Key Takeaways
- Breaks binary test-suite limitations in code agent RL via sum-preserving groupwise advantage redistribution
- Validated at massive industrial scale across 310B (MiMo-V2.6-Flash) and 1.02T parameter (MiMo-V2.6-Pro) backbones
- Restricts unnecessary trajectory length growth by 36% while reducing gradient update variance by 42.3%