Benchmark1d ago
Einsia and Tsinghua Release WorkflowArena, a Multi-Agent Orchestration Benchmark: 145 Long-Horizon Tasks, 25 Claude Code and Codex Configurations, and Even Top-Ranked Opus 5.5 Fully Resolves Only 36.95%
On November 10, Einsia AI, with Tsinghua University and others, released WorkflowArena, a benchmark that tests whether a model can split a task, dispatch worker agents, write the workflow, and integrate the results on its own. It has 145 long-horizon tasks across 8 professional domains, scored by hidden verifiers, with every delegation recorded. The paper tests 25 configurations of 9 models in Claude Code and Codex: the best, Claude Opus 5.5 at max effort, fully resolves only 36.95% of tasks on average, the best SWE rate is 29.41%, and law never exceeds 9.09%. Claude models delegate rarely but in large groups, GPT models often and in small units, and workflows written by Opus raise weaker executors by 6.41 points on average. Code and paper are public.
Benchmark21h ago
Google DeepMind and Artificial Analysis Launch MKA, a Multimodal Knowledge Agent Benchmark of 200 Real Visual Tasks: Claude Fable 5.2 Leads at 57%, Gemini 4 Carbon Scores 54% at About One-Fifth the Cost per Task
On November 10, Google DeepMind and Artificial Analysis launched MKA (Multimodal Knowledge Agent), a benchmark that tests whether agents can read visually dense files such as architectural drawings, engineering schematics, and clinical scans and deliver slide decks, spreadsheets, reports, and marked-up images. Its 200 private tasks span 10 industries and 83 occupations, with about 70 expert rubric checks per task. Artificial Analysis tested 21 models at launch: Claude Fable 5.2 (max) leads with a Climb WSR of 57%, followed by Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%. Fable averages $31.97 per task versus $6.29 for Carbon. Ten public tasks are on Hugging Face.
Benchmark1d ago
Artificial Analysis: Claude Fable 5.2 Takes #1 on the AA-3D Interactive 3D Benchmark at 1271 Elo, 67 Points Clear, and Leads the VIALS-AA Scientific Image Reasoning Benchmark at 67%
Independent evaluator Artificial Analysis published two new results on November 10. On AA-3D, which tests how well a model acting as an agent builds interactive three.js web scenes, Claude Fable 5.2 (Max) scores 1271 Elo, 67 points ahead of second-place Claude Sonnet 5.5 (1204); its 95% confidence interval of 1246 to 1297 does not overlap any other model, so the lead is outside the margin of error. It is 132 points above Fable 5.1 (1139) and ahead of the top OpenAI and Google entries, GPT-6 Astra (1168) and Gemini 4 Argon (1147). On VIALS-AA, a life-science image reasoning benchmark, Fable 5.2 scores 67%, ahead of previous leader Gemini 4 Carbon (64%) and GPT-6 Astra (63%). AA also found opposite strategies: Fable averages about 37 tool calls per task, Astra about 8.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.