Latest news, beside the leaderboard

Tool-using agents execute consequential modifications to external systems, yet nominal task success frequently conceals unverified actions executed without prior evidence. Researchers from the National University of Singapore (NUS) present a rigorous empirical study on the breakdowns along the evidence-to-action chain (arXiv:2610.07753). Evaluating ten model-harness configurations reveals a striking paradox: high static action assessment accuracy routinely masks brittle interactive execution. Failures overwhelmingly originate prior to execution: agents terminate prematurely before collecting requisite evidence, or initiate state-altering actions before foundational proofs are established. The authors formulate SafeActBench, spanning 656 rigorous cases across six operational domains and five protocols progressing from static judgment to complex multi-action workflows. Supported by a provenance-bound Evidence Ledger and deterministic trajectory evaluator, findings demonstrate that while single-step execution is stable once evidence is verified, multi-action workflows frequently collapse under unresolved prerequisites and truncated state transitions.

Continual skill evolution allows LLM agents to autonomously accumulate, refine, and reuse procedural knowledge across multi-turn interactions without updating model parameters. However, prevailing experience-driven reflection mechanisms suffer from two fundamental weaknesses: they decouple synthesized guidance from concrete behavioral traces, and they rely on monolithic global validation that blindly discards valid local corrections whenever an entire workflow fails. Researchers from Tsinghua University and partner institutions introduce EVISKILL (arXiv:2610.05030), an evidence-grounded framework anchoring procedural adaptation to deterministic execution traces. EVISKILL structures raw observations into Replayable Evidence Cards and synthesizes skill edits with explicit contextual lineage. A targeted replay engine verifies edits through isolated re-execution before cross-epoch refinement. Across three interactive benchmarks spanning six LLM backbones, EVISKILL consistently outperforms state-of-the-art reflective baselines, demonstrating durable procedural accumulation.

Agentic reinforcement learning (RL) disaggregates centralized policy training from large-scale interactive environment rollouts, necessitating frequent cross-cluster policy weight synchronization (refit). Transferring a full 1T-parameter checkpoint across cloud regions currently consumes 87.5 minutes, idling massive rollout compute clusters. NVIDIA researchers introduce NeMo-DCR (Delta-Compressed Refit, arXiv:2610.08430), a bit-exact synchronization architecture that transmits only model deltas while guaranteeing receivers reconstruct identical parameter bits. Exploiting empirical measurements showing only ~1% of BF16 weights alter value bits per step, NeMo-DCR maps training shard changes into canonical coordinates via fixed affine projections and utilizes compressible XOR masks to guarantee exact bit-level parity. Streaming deltas over relay trees and committing changes via atomic joint commits, NeMo-DCR achieves 12x to 40x speedups for models spanning 30B to 1T parameters, slashing 1T cross-region refit latencies from 87.5 minutes to 150 seconds.

At Team '26 Europe, Atlassian introduced AMP, a framework giving agents an assigned identity, scoped authority, shared Teamwork Graph context and reviewable output, and made a rebuilt Atlassian MCP generally available with 220+ tools, OAuth 2.1/PKCE and tiered read/write/destructive scopes.

On 2026-10-06 Google officially launched Nano Banana 2.1 and marked it GA in the Gemini API changelog as gemini-nano-banana-2.1. It is based on Gemini 3.6 Flash and is rolling out across the Gemini app, AI Studio, Search AI Mode, Flow, Stitch, Google Ads and Gemini Enterprise Agent Platform. Image output is priced at $30 per 1M tokens (Nano Banana 2: $60), which works out to $0.0336 per 1K image and $0.0504 per 2K image, about half the old price. The model card's side-by-side human eval puts text-to-image Elo at 1050, ahead of Nano Banana 2 (990) and Nano Banana Pro (935). gemini-3.1-flash-image is now deprecated, with no shutdown date announced. This confirms the silent Flow rollout we reported earlier.

On 2026-10-06 GitHub made stacked pull requests generally available on all github.com plans, with GitHub Enterprise Server support coming in a later release. Since the public preview, repos using stacks have merged 9% more code than their peers, and more than two-thirds of the top 1% of repos now use them, with a 5% faster time-to-merge. New in GA: approvals are kept for unchanged code after a rebase, replacement commits are signed automatically, a whole stack enters the merge queue as one merge group, stacks retarget automatically when their base branch is deleted, and stack auto-merge is rolling out over the coming weeks. The gh-stack CLI extension (v0.2.0) now supports Git worktrees, and `gh skill install github/gh-stack` teaches AI coding agents how to split large changes into stacked PRs.

On 2026-10-06 Anthropic merged Project Glasswing and the original Cyber Verification Program into one three-tier program: Defense Access (SOC, incident response, malware reversing, vuln validation), Red Team Access (authorized pentesting) and Specialized Access (safety-critical systems such as power grids, flight systems and telecom, vetted with the US government). All tiers get Claude Opus 5.5, Sonnet 5.5 and Mythos 5.1 with reduced cyber blocking. On CyScenarioBench, GA models were blocked on the first prompt of every task, Defense tier blocked 46/50 trials, and Red Team tier had zero blocks and completed 34/50 (matching the 67.6% no-safeguard rate). Glasswing partners reported at least 129,000 verified vulnerabilities from April to July 2026.

FeSens/openTPU (Apache-2.0) hit the Hacker News front page (250+ points, 300+ comments). Reusing the AI-driven method from auto-arch-tournament (AI-designed RISC-V cores), the author had AI agents iterate on an inference accelerator; one repo holds SystemVerilog RTL, the ISA, a bit-exact simulator, a kernel language/compiler and a profiler. On a Kintex-7 xc7k480t PCIe card it runs Qwen3, Qwen3.5, Gemma 4, Phi-4-mini and others with real weights, token-for-token identical to the simulator; LFM2.5-230M decodes at 85.8 tok/s in 4-bit and Qwen3.5-35B-A3B reaches 3.95 tok/s with host-streamed experts.

Large Language Model (LLM) coding agents equipped with bash terminals frequently collapse when deployed across massive enterprise codebases. Centralized agents struggle to reconstruct project states fragmented across multi-language source trees, build configs, and test suites, precipitating context window exhaustion and catastrophic semantic drift. Researchers from UIUC introduce Dev-Primitives and the HERMES harness engineering framework (arXiv:2610.07832). Dev-Primitives transform passive software components into active agentic nodes by pairing each repository artifact with a resident LLM endowed with natural-language reasoning, inter-component communication channels, and localized self-modification primitives. Coordinated via a dependency-aware dynamic activation scheduler and execution-guided fault localization, HERMES boosts task performance across four standard software engineering benchmarks by 12.4% over baseline harnesses. Crucially, powering HERMES with lightweight Qwen3-8B Dev-Primitives brings performance within 4.5% of homogeneous GPT-5.6 Sol configurations while slashing inference token overhead by 26.2% on Terminal-Bench 4.0.

When an external tool or search endpoint repeatedly returns irrelevant noise, a rational agent should cease querying and act upon available knowledge. A comprehensive study by Nanyang Technological University (NTU) and A*STAR (arXiv:2610.06191) uncovers a profound behavioral failure mode: while LLM agents correctly identify tool outputs as useless 97% to 100% of the time, they systematically fail to translate those negative evaluations into stopping decisions, entering persistent query loops. Analyzing seven frontier open-source and proprietary models under controlled tool failure modes, the authors show that prompt warnings regarding token budgets, call costs, or explicit stopping guidelines fail to enforce rational early exits—often prompting smaller 7B-8B models to burn calls until hit by hard deadlines. Evidence-grounded stopping emerges reliably only when the execution harness enforces a programmatic policy: mandating final generation after five consecutive self-judged useless queries. This integration rule raises task completion rates across every evaluated model during source failures, confirmed via a 300-question pre-registered replication trial.

The executable harness surrounding a GUI model governs how multimodal visual observations are assembled, how mouse/keyboard actions are executed, and how error recovery and termination gates operate. Automatically evolving this harness with frozen base models presents coupled challenges: grounding error diagnosis in visual UI transitions, attributing failures amid execution variability, and translating multi-task failure modes into reliable runtime code edits. Researchers introduce GUI-HARVEST (arXiv:2610.00948), an evidence-driven harness optimizer. GUI-HARVEST synchronizes model intentions with before-and-after screenshots, treats repeated rollouts as joint evidence units, and abstracts recurring failure patterns into bounded Python code edits applied directly to the harness. Evaluated on OSWorld-Verified across six general and GUI-specialized backbones, Qwen3-VL-32B-Instruct improves by 12.33 points. Remarkably, transferring the evolved harness to WindowsAgentArena without further optimization elevates GPT-5 by 13.87 percentage points at 50 steps, decisively outperforming Self-Harness and Meta-Harness. Code is available at github.com/GaryYang12345/GUI-HARVEST.

On 2026-10-06 Sierra co-founders Bret Taylor and Clay Bavor announced Personal Agent Protocol, an open standard Meta and Sierra are developing with Genesys, Instinct, Rocket, Shopify, Stripe and Walmart. Built on OAuth, it lets a personal agent discover what a business offers, start a guest session, then act with user-granted read-only or write access via the company's website, APIs (MCP/OpenAPI) or its own agent. A v0.1 spec, design workshops and a reference implementation are planned for later in October; payments are a future extension.

On 2026-10-06 OpenAI published [Sharing AI progress in mathematics](https://openai.com/index/sharing-ai-progress-in-mathematics/) and released [openai/math](https://github.com/openai/math) (Apache-2.0): 722 manuscripts in 372 result families produced by an unreleased internal frontier model, drawn from an evaluation of ~4,000 open problems at an average of ~3 hours of ChatGPT Pro thinking compute per result, with 10 abridged reasoning summaries and Lean formalizations for many results. OpenAI notes unformalized results may have issues; none are peer reviewed yet.

Per the [OpenAI API changelog](https://developers.openai.com/api/docs/changelog), the [Decisions API](https://developers.openai.com/api/docs/guides/decisions) entered public beta on 2026-10-06 with gpt-6-luna: a dedicated POST /v1/decisions endpoint takes text/images plus a list of questions and returns typed predicate, choice or score answers with probabilities, about 10x faster than the Responses API. Billing is input-only at $0.10 per 1M tokens (no output or cache charges); ZDR and HIPAA are supported and GA is expected within weeks.

Conversational memory for LLM agents traditionally divides into two opposing camps: expensive pipeline extraction that distills dialogues into structured factual memory stores versus raw conversational turn retrieval. Literature has reported contradictory findings. Researchers conducted a pre-registered, double-blind controlled study on held-out LoCoMo and LongMemEval benchmarks (arXiv:2609.34227). The investigation reveals that under constrained context budgets, selecting raw conversational turns via a lightweight typed decision model (Jev) is statistically non-inferior to full LLM fact extraction pipelines (95% one-sided bound -3.0 points vs. -5 margin)—while reducing write-time computational costs by a massive 3,061 times. Furthermore, the authors resolve literature contradictions by proving that reranking gains scale inversely with token budgets, while aggressive reranking degrades an agent's correct abstention reliability. All study protocols, data, and code are open-sourced at github.com/ris3abh/Engram.

Dense retrieval serves as the default paradigm for searching unstructured enterprise corpora in modern RAG systems. However, dense retrieval is fundamentally bound to surface-level semantic similarity, failing when queries require multi-hop exploration, dynamic hypothesis reformulation, and causal synthesis. Researchers from NVIDIA NeMo-Retriever present an in-depth investigation into Agentic Retrieval (arXiv:2610.05750), coupling LLM ReAct loops with dense and lexical retrievers. Evaluating across challenging benchmarks including ViDoRe v3 and BRIGHT, agentic retrieval improves nDCG@10 by 8.7 points using the identical underlying embedding model. Crucially, NVIDIA transparently quantifies the operational trade-offs: agentic searches average 107.4 seconds per query (versus 0.67 seconds for conventional retrieval) and consume an average of 764.1K input tokens and 5.8K output tokens per query. This landmark study provides critical empirical benchmarks for enterprise architectures balancing retrieval depth against inference budgets (github.com/NVIDIA/NeMo-Retriever).

Large language models frequently encounter a probability mass dilemma in multi-step reasoning: while the correct sequence might possess a higher individual probability than any single wrong candidate, the vast ocean of incorrect paths collectively dominates the posterior distribution. Sequence-level power sampling sharpens distributions toward the mode by raising probabilities to exponents greater than one, but requires generating and scoring dozens of rollout candidates at runtime. Researchers from USC and collaborators introduce On-Policy Power Distillation (OPPD, arXiv:2610.06804), an algorithm that teaches models to output power-sharpened solutions in a single forward pass without inference-time search. By running an on-policy Sequential Monte Carlo (SMC) teacher-student distillation loop, OPPD boosts single-generation accuracy by 23.0 points on MATH500 and 27.3 points on GSM8K without external reference solutions. Crucially, OPPD outperforms verified-reward GRPO on MATH500 (+3.8), GSM8K (+4.0), and AIME (+5.4) under identical compute budgets, while cross-generalizing to raise HumanEval code generation by 5.3 points. The repository is available at github.com/ArminAzizi98/OPPD.

OpenAI's first vertical-SaaS computer-use collaboration: with Ironclad it defined 11 legal, procurement and commercial tasks, let models practice in hosted Ironclad environments, and used RL on synthetic tasks. GPT-6 Astra scores 55.0% mean rubric vs 41.6% for GPT-5.6 Sol, with estimated time per attempt down from 37.0 to 19.2 minutes; an internal model reaches 63.7%. OpenAI is inviting more software companies to partner.

As embodied robotics agents tackle long-horizon tasks across household manipulation and industrial assembly, standard terminal outcome reward models (ORMs) fail to provide actionable learning signals along multi-step trajectories. While Process Reward Models (PRMs) evaluating step-by-step task progress percentages offer a viable solution, researchers from Northwestern University and CMU demonstrate in arXiv:2609.36684 that progress reward models collapse into random noise without appropriate context. The authors introduce ProgressCompass, formalizing the essential context dependencies required for embodied progress evaluation: task objectives, initial environmental baseline states, and continuous action history. Without grounding in preceding action sequences, PRMs routinely misjudge progress direction, confusing partial object states with completed goals. ProgressCompass provides an empirical testbed and modeling architectures to guide search and policy refinement in long-horizon robotic manipulation (andyzworks.github.io/progresscompass/).

Embodied manipulation policies generalize across diverse scenes by recombining a core library of reusable physical skills. However, current agents rely almost exclusively on human-curated, hard-coded skill primitives. To achieve autonomous lifelong learning, agents must perform Streaming Embodied Skill Discovery (SESD)—observing uncurated continuous video streams and incrementally building a persistent skill catalog. Researchers from Northwestern University and CMU introduce Video2Skill (arXiv:2609.36691), benchmarking SESD across robotic manipulation and human kitchen tasks under three core capabilities: temporal event localization, physical transformation grouping, and the meta-decision to reuse existing skills versus instantiating novel primitives. Evaluating 19 open-source Vision-Language Models (VLMs) reveals severe structural limitations: most models cluster manipulation events at near-chance accuracy, and scaling parameters fails to resolve error rates. Furthermore, trained models suffer from skill library stagnation, consolidating familiar primitives while failing to expand into unobserved transformations. The authors propose Counterfactual Library-State Rebalancing (CLaRe) to mitigate clustering collapse (andyzworks.github.io/video2skill/).