AgSpec Accelerates Coding Agent Pipelines by 4.7x: Retrieval-Based Speculative Decoding Outperforms EAGLE-3
Coding agents frequently reproduce existing code, error logs, and multi-turn refactoring traces, making retrieval-based speculative decoding (SD) an ideal acceleration paradigm. However, conventional retrieval drafters fail in agent workflows: candidate code on disk diverges from the agent's emission formats (e.g., diff blocks, escape characters), and static draft lengths ignore inter-agent turn drift. KAIST researchers introduce AgSpec, a speculative decoding framework tailored for coding agents. AgSpec constructs format-aligned corpora across session trajectories, workspaces, and global repositories, coupling offline draft caps with online feedback-adaptive draft length control. Across repository-level multi-agent benchmarks, AgSpec outperforms five retrieval drafters and EAGLE-3, boosting generation throughput up to 4.76x.
Key Takeaways
- Tailors retrieval speculative decoding to coding agent pipelines via emission-aligned triple corpora and adaptive length control
- Achieves up to 4.37x speedup at batch size 1 and 4.76x speedup at batch size 16 on repository-level multi-agent benchmarks
- Outperforms EAGLE-3 across primary settings without requiring dedicated neural draft models or additional GPU VRAM
