AI Coding Live News

Published in arXiv:2610.25834, FrontierRefactor establishes the industry's first evaluation benchmark targeting autonomous coding agents on multi-scale, performance-oriented codebase refactoring. Unlike traditional benchmarks like SWE-bench that measure only functional pass rates, FrontierRefactor spans seven programming languages and 60 Amdahl-qualified tasks ranging from kernels to full repositories (10^1 to 10^4+ LOC). Initial evaluations reveal that over 41% of agent patches pass functional equivalence suites yet yield zero verified speedup—with 19.3% triggering severe runtime performance regressions, highlighting the vital need for multi-turn profiling feedback.

Zoom

Mainstream coding agent benchmarks like SWE-bench frame evaluation around issue-derived tasks: feeding an agent a bug description and expecting an end-to-end patch. But does this reflect genuine developer practices? JetBrains Research analyzes 4,782 production sessions of professional software engineers inside JetBrains IDEs (arXiv:2610.09633). For long interactive sessions (>=3 turns, comprising 33% of production traffic), user requests span a diverse mix of intent types—project codebase Q&A, strategic planning, code review, refactoring, and test execution—with users frequently transitioning between types mid-session. To align benchmarks with real developer task flows, JetBrains introduces SWE-TaskFlow, an approach that transforms issue-based benchmarks via prompt-splitting and verifiable repository QA into calibrated multi-stage workflows guided by a TaskFlow Alignment Score (TFAS). A pilot study across 700 SWE-Bench Pro tasks demonstrates that sequential multi-step interaction protocols nearly double agent inference expenditure without stable improvements in resolve rates, proving that the interaction protocol itself is a critical, previously unmeasured dimension of agent evaluation.

Zoom

Coding agents frequently reproduce existing code, error logs, and multi-turn refactoring traces, making retrieval-based speculative decoding (SD) an ideal acceleration paradigm. However, conventional retrieval drafters fail in agent workflows: candidate code on disk diverges from the agent's emission formats (e.g., diff blocks, escape characters), and static draft lengths ignore inter-agent turn drift. KAIST researchers introduce AgSpec, a speculative decoding framework tailored for coding agents. AgSpec constructs format-aligned corpora across session trajectories, workspaces, and global repositories, coupling offline draft caps with online feedback-adaptive draft length control. Across repository-level multi-agent benchmarks, AgSpec outperforms five retrieval drafters and EAGLE-3, boosting generation throughput up to 4.76x.

Zoom