Tracing the Thoughts of a Coding Agent Playing ARC-AGI-3: White-Box Lessons for Continual Learning
Continual learning without weight fine-tuning represents a frontier in LLM agent research. When an agent acts purely by writing and executing Python scripts inside an execution sandbox, every thought, hypothesis, and plan leaves a verifiable trace on disk. In a newly released study (arXiv:2610.11450), researchers subject coding agents to ARC-AGI-3—a suite of instruction-free abstract visual reasoning environments where each sequential level introduces novel causal dynamics. Developing a fine-grained measurement protocol, the authors trace thoughts across file artifacts across seven full evaluation runs spanning three foundation backbones from two major model families. The empirical findings reveal that scripts written for one level are almost never invoked in subsequent levels (only 33 of 630 cross-task references survive), as agents deliberately abandon 74% of scripts at task boundaries to purge level-specific overfitted state while rewriting core rules. Notes are maintained in an append-only fashion, with conflicting claims arbitrated against raw execution logs to achieve selective rather than catastrophic forgetting. Crucially, the study identifies carrying hard-coded variables across task boundaries as the single most devastating failure mode of autonomous agents.
Key Takeaways
- Presents the first white-box thought-tracing audit of coding agents continually learning across ARC-AGI-3 environments
- Finds agents deliberately abandon 74% of scripts at task shifts to purge state, settling append-only notes against immutable logs
- Identifies carrying hard-coded constants across task boundaries as the single most catastrophic agent failure mode