Einsia and Tsinghua Release WorkflowArena, a Multi-Agent Orchestration Benchmark: 145 Long-Horizon Tasks, 25 Claude Code and Codex Configurations, and Even Top-Ranked Opus 5.5 Fully Resolves Only 36.95%
On November 10, Einsia AI, with Tsinghua University and others, released WorkflowArena, a benchmark that tests whether a model can split a task, dispatch worker agents, write the workflow, and integrate the results on its own. It has 145 long-horizon tasks across 8 professional domains, scored by hidden verifiers, with every delegation recorded. The paper tests 25 configurations of 9 models in Claude Code and Codex: the best, Claude Opus 5.5 at max effort, fully resolves only 36.95% of tasks on average, the best SWE rate is 29.41%, and law never exceeds 9.09%. Claude models delegate rarely but in large groups, GPT models often and in small units, and workflows written by Opus raise weaker executors by 6.41 points on average. Code and paper are public.
Key Takeaways
- Scale: 145 long-horizon tasks across 8 domains and 17 task forms (software engineering, Lean formal proofs, law, finance, science, operations, content, 3D, and more), testing 25 configurations of 9 models
- Full resolution: Opus 5.5 (max) 36.95%, Opus 5.5 (high) 34.11%; best math rate 95.83%, best SWE rate 29.41%, while law and finance never exceed 9.09% and 18.75%
- Cost: Opus (high) uses about 7.36M tokens and $6.66 per task; GPT-6.1 Sol (low) gets 27.59% at $3.61; GPT-6 Astra (xhigh) gets 30.80% but costs $29.26 (estimated from API prices)
- Delegation style (high effort): Opus / Sonnet make 0.86 / 0.76 workflow calls per task with 5.55 / 6.14 workers each; Astra / Sol make 2.09 / 1.94 calls with only 1.72 / 1.69 workers
- With the executor held fixed, Opus-written workflows raise the continuous scores of 4 weaker executors by 6.41 points on average, while Astra-written workflows help far less and lower two of them
