AI Coding Live News

On October 30, Microsoft's Mage team released Mage-VL-Cowork on Hugging Face under Apache-2.0: a 4B multimodal computer-use agent in which a single model decides at every step whether to act through screenshots and GUI clicks (computer_use) or through bash and Python in a terminal, with no external orchestrator. Microsoft reports a two-run average of 66.0% on OSWorld-Verified (361 tasks, 50-step budget), above Qwen3.6-27B at 63.9% but below the 9B SCALECUA-Qwen3.5-9B at 68.7%.

Zoom

Alibaba Tongyi Labs (Tongyi-MAI) has published and open-sourced OpenGCUA (arXiv:2610.22200), establishing a fully open software stack for Generalist Computer Use Agents. Moving beyond brittle single-click GUI paradigms, OpenGCUA introduces a 'Code-Execution-as-Action' framework combining visual screen perception, accessibility trees, bash scripts, and Playwright APIs into unified programmatic execution blocks. Evaluated across eight frontier models on OSWorld-v2, OpenGCUA boosted mean scores by +19.6 points while cutting task steps by 53.0%, releasing an Apache-2.0 macOS menu-bar harness and Cross-Device SDK.

Zoom

Researchers introduced 'Evolving Agent Interfaces' alongside the Agent S4 meta-framework in arXiv:2610.28601, removing the fundamental bottleneck of computer-use agents (CUAs). Conventional CUAs are forced to operate through human-designed graphical user interfaces—such as clicking spreadsheet cells one by one—inducing excessive action horizons and high failure rates. The new framework allows meta-agents to inspect live applications in scratch environments, dynamically synthesizing, testing, and evolving programmatic API wrappers and tool repositories on the fly. On OSWorld V2 and ScienceBoard, an open-weights 27B model raised task success from 55.03% to 58.58%, rivaling proprietary frontier models at a fraction of the cost.

Zoom

Alibaba's Tongyi-MAI team has open-sourced two foundational frameworks for generalist computer-use agents: OpenGCUA (arXiv:2610.22200) and MAI-Gym (arXiv:2610.20939). By unifying GUI interaction with programmatic code execution and harness-to-model decision transfer, OpenGCUA elevates scores on OSWorld-v2 across all eight tested models—including GPT-6 Astra and Claude Opus 5—by +19.6 points while cutting steps by 53.0%. Meanwhile, MAI-Gym scales online RL to 20,000 parallel instances, training MAI-Gym-9B to 78.4% on MobileWorld and 76.2% on OSWorld-Verified.

Zoom

Computer-Use Agents (CUAs) designed to operate graphical desktop environments through virtual keyboard and mouse interactions are traditionally evaluated using outcome-based functional verifiers that assess only the final state after hundreds of operational turns. This black-box approach obscures whether failures stem from syntax typographical errors, GUI bounding-box misalignments, or intermediate state drift, preventing actionable diagnostic insights. NVIDIA Research introduces OSWorld-Pro (arXiv:2609.24890), the first comprehensive process-based evaluation suite for computer-use agents. Spanning 300+ complex desktop workflows decomposed into 2,800+ sequentially dependent subgoals grounded in 67,000+ expert human annotations, OSWorld-Pro deploys calibrated LLM judges to track incremental subgoal completion. Evaluations reveal that OSWorld-Pro presents a formidable challenge to state-of-the-art models: frontier models like Claude Opus 5 achieve only 75.7% (compared to 83.4% on standard OSWorld), systematically isolating critical procedural failure patterns such as exploratory irrelevant actions and click grounding drifts.

Zoom

Modern software engineering transcends isolated code edits: engineers run applications, interact with GUI interfaces, and visually inspect rendering feedback to diagnose errors and verify changes. CUA-SWE bridges coding agents and computer-use agents by introducing the first unified benchmark and environment for visual software engineering. Spanning four engineering domains, CUA-SWE challenges agents to interleave code/config modifications, terminal execution, GUI interactions, and screenshot inspections, with deterministic test suites evaluating behavioral correctness.

Zoom

Existing Computer-Use Agents (CUAs) face a dilemma: relying strictly on graphical user interface (GUI) interactions is slow and error-prone, while building per-application API tools demands unsustainable engineering and lacks generalizability. Researchers from Zhejiang University introduce HybridCUA, combining the UI versatility of GUIs with the execution speed and determinism of the Command Line Interface (CLI). Supported by the newly constructed HybridCUA-8K dataset (5K hybrid trajectories and 3K RLVR tasks), HybridCUA employs a two-stage training paradigm: supervised fine-tuning across interleaved GUI-CLI modalities, followed by CLI-aware reinforcement learning that rewards reliable, selective shell execution. On the OSWorld benchmark, HybridCUA-9B achieves 53.6% accuracy, improving over the base model by 14.8 percentage points, while delivering a 4.0-point gain on WindowsAgentArena.

Zoom