HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention Scoring 83.3% on Held-Out Tasks
Researchers introduced HERA in arXiv:2610.06563, addressing a critical reliability gap in autonomous LLM agents: the inability to detect task infeasibility and reliably abstain from blind, destructive tool execution. Rather than tuning harnesses against static benchmarks that quickly saturate, HERA establishes a harness-environment co-evolution framework that mutates execution environments to expose emergent failure modes while adapting the scaffold. Evaluated across 19 frontier LLMs using the newly released hera-bench, HERA elevates agentic abstention success from 61.7% to 83.3% and paired task success from 48.3% to 70.0%.

Key Takeaways
- HERA introduced in arXiv:2610.06563 alongside hera-bench, formalizing the evaluation of agentic abstention in executable tool environments
- Pioneers harness-environment co-evolution, continuously generating matched feasible-infeasible mutations to probe blind-spots
- Increases agentic abstention success from 61.7% to 83.3% and paired success from 48.3% to 70.0% on held-out tasks
- Demonstrates seamless transferability across 19 frontier foundation models, slashing API expenditure and avoiding destructive rogue actions

