On November 10, Einsia AI, with Tsinghua University and others, released WorkflowArena, a benchmark that tests whether a model can split a task, dispatch worker agents, write the workflow, and integrate the results on its own. It has 145 long-horizon tasks across 8 professional domains, scored by hidden verifiers, with every delegation recorded. The paper tests 25 configurations of 9 models in Claude Code and Codex: the best, Claude Opus 5.5 at max effort, fully resolves only 36.95% of tasks on average, the best SWE rate is 29.41%, and law never exceeds 9.09%. Claude models delegate rarely but in large groups, GPT models often and in small units, and workflows written by Opus raise weaker executors by 6.41 points on average. Code and paper are public.
Key Takeaways
- ✓Scale: 145 long-horizon tasks across 8 domains and 17 task forms (software engineering, Lean formal proofs, law, finance, science, operations, content, 3D, and more), testing 25 configurations of 9 models
- ✓Full resolution: Opus 5.5 (max) 36.95%, Opus 5.5 (high) 34.11%; best math rate 95.83%, best SWE rate 29.41%, while law and finance never exceed 9.09% and 18.75%
- ✓Cost: Opus (high) uses about 7.36M tokens and $6.66 per task; GPT-6.1 Sol (low) gets 27.59% at $3.61; GPT-6 Astra (xhigh) gets 30.80% but costs $29.26 (estimated from API prices)
- ✓Delegation style (high effort): Opus / Sonnet make 0.86 / 0.76 workflow calls per task with 5.55 / 6.14 workers each; Astra / Sol make 2.09 / 1.94 calls with only 1.72 / 1.69 workers
- ✓With the executor held fixed, Opus-written workflows raise the continuous scores of 4 weaker executors by 6.41 points on average, while Astra-written workflows help far less and lower two of them
Key Decision Metrics at a Glance
Heavy Claude Code use: compare subscription limits and API bills
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Core Background and Industry Pain Points
Agent tools such as Claude Code and Codex now let a model spawn its own worker agents and write the workflow that coordinates them. Existing benchmarks either check only whether the final task is done or test coordination inside a structure fixed in advance, so neither shows how well a model organizes the work. On November 10, Einsia AI, with Tsinghua University and others, released WorkflowArena to evaluate workflow orchestration on its own.
Architecture Highlights and Underlying Mechanisms
Each task provides a starting workspace and a deliverable built from many interdependent parts, such as clause analyses that must agree across a whole contract portfolio, or proofs that build on earlier results. The model decides whether and how to delegate. Every delegation passes through a recorded interface, and time spent organizing comes out of the same budget as the work. Hidden verifiers specifically catch submissions whose parts are each correct but never reconciled.
Authoritative Benchmarks and Measured Comparisons
These figures come from the paper. The best configuration, Opus 5.5 at max effort, fully resolves 36.95% of tasks on average, and no configuration leads every domain. Opus at high effort reaches 34.11% for $6.66 per task, while GPT-6 Astra at xhigh reaches 30.80% for $29.26. Claude models in Claude Code delegate rarely but in large groups, and GPT models in Codex delegate often in small units. Workflows written by Opus raise weaker executors by 6.41 points on average, which suggests that orchestration is a different skill from completing the task alone.
Developer Adoption and Getting Started
The tasks, harness, and paper are public on GitHub (Einsia/WorkflowArena) and arXiv (2611.12856), so you can reproduce results on your own agent system. For teams using multi-agent tools like Claude Code or Codex, the results suggest letting a strong model split hard long-horizon tasks and write the workflow, then handing execution to cheaper models. The effort setting matters a lot for Claude models and is worth tuning separately.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.