On 2026-10-12 LightOn AI released ColGrep-Agent-2B (fine-tuned from MiniCPM5-2B, Apache-2.0): a local 2B code-localization agent that searches with ColGrep multi-vector retrieval plus a read-only terminal, then returns file paths and line ranges to a frontier coding agent. On 52 code-search questions across 13 unseen repos (624 Claude Code / Sonnet 5.5 sessions), delegating search cut API tokens ~34.3% and cache-aware cost ~40% vs grep, with 80.8% vs 83.3% accuracy (not statistically significant). On SWE-bench Lite localization it posted 19.92 line F1—best among models evaluated in the same ColGrep harness, including several >10× larger. Ships brew/cargo/GGUF installers and hooks for Claude Code, Codex, and OpenCode.
Key Takeaways
- ✓Official: HF Blog — ColGrep-Agent (2026-10-12); model lightonai/colgrep-agent-2B; repo lightonai/next-plaid
- ✓Vendor Claude Code + Sonnet 5.5 study (52×3): −34.3% tokens vs grep (95% CI 28–40), ~−40% cache-aware cost (7.3¢→4.4¢/q); accuracy 80.8% vs 83.3%
- ✓Localization: SWE-bench Lite file F1 76.18 / line F1 19.92; Multi-SWE-bench Flash file F1 62.57 / line F1 15.93 (5-seed avg)
- ✓Train: MiniCPM5-2B → SFT (Qwen3.5-27B teacher) → GRPO 500 steps; tools: colgrep / read-only terminal / finish; 10-turn budget
- ✓Install:
brew install lightonai/tap/colgrep && colgrep --install-agent; default GGUF ~2.5GB; hooks for Claude Code / Codex / OpenCode
Key Decision Metrics at a Glance
Heavy Claude Code use: compare subscription limits and API bills
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Context
Frontier coding agents often burn turns grepping and opening files before they can patch anything; each turn resends growing context to the API. LightOn’s 2026-10-12 post splits that exploration to a local specialist: ColGrep-Agent-2B searches with ColGrep multi-vector retrieval plus a read-only terminal, then returns path:START-END snippets to the frontier model. The goal is fewer billed exploration turns—not replacing end-to-end patching—and keeping source local during search.
Architecture
Hooks (colgrep --install-claude-code and siblings) suggest—not force—search tools (hook-mode agent|hybrid|colgrep). ColGrep’s LateOn-Code-edge (~17M multi-vector) can match behavioral phrasing to differently named symbols. The agent has three tools: colgrep, read-only terminal, and finish, with a 10-turn budget identical to training. Training starts from MiniCPM5-2B, SFT on filtered Qwen3.5-27B teacher traces, then GRPO for localization with file/line F1 rewards and turn/grep penalties.
Benchmarks
Vendor Claude Code study (13 public 2026 repos, 52 questions, 624 Sonnet 5.5 sessions): ColGrep-Agent used 93k vs 142k tokens (−34.3%) and ~4.4¢ vs 7.3¢ cache-aware cost vs grep; accuracy 80.8% vs 83.3% (CI crosses zero—authors flag non-significance); wall time ~21.6s vs 13.0s. Balanced mode saved ~21% tokens at matched accuracy. Localization suite (same ColGrep harness): SWE-bench Lite file/line F1 76.18 / 19.92; Multi-SWE-bench Flash 62.57 / 15.93. Line F1 beat several >10× larger models in-table. Treat Claude Code numbers as LightOn’s protocol on search questions, not full SWE end-to-end.
Developer path
brew install lightonai/tap/colgrep && colgrep --install-agent (or shell/PowerShell/cargo); ~2.5GB default GGUF, no Python/API key. colgrep init then colgrep --agent "…". Install coding-agent hooks and prefer agent mode on large/cross-folder work, hybrid when latency matters. Paper: arXiv:2610.05099.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.