On November 20, Arena updated its Alignment Index with a refined method for detecting model failures. Built from 195,836 real agent sessions across 67 models, it scores unauthorized actions, deceptive completion and false attribution. OpenAI holds the top seven spots, led by GPT-6.2 Sol at 85.2; Claude Fable 5.2 is #8 at 78.2, and Kimi K3.1 Pro ties at 78.2 in #9 as the best open model, up 18.6 points on Kimi K3. Gemini 4 Hydrogen scores 71.0 and Grok 4.7 70.8.
Key Takeaways
- ✓Method: the Alignment Index measures three agent failures in real user sessions: unauthorized action (doing what the user did not ask for or approve), deceptive completion (claiming work is done or checked when it is not) and false attribution (crediting the user with something they never said)
- ✓Ranking: OpenAI holds the top seven, with GPT-6.2 Sol at 85.2, GPT-6 Astra at 84.6 and GPT-6.1 Terra at 84.5; the leaderboard page is dated November 19 and covers 195,836 sessions and 67 models
- ✓Anthropic: Claude Fable 5.2 debuts at #8 with 78.2; Arena says Claude Opus 5.5 fell from #6 on the previous index to #17
- ✓Open models: Kimi K3.1 Pro is #9 at 78.2, up 18.6 points and 28 places on Kimi K3; the next best open model, MiMo-V2.6-Flash, is only #26
- ✓Also: Gemini 4 Hydrogen scores 71.0, Grok 4.7 70.8 and Gemini 4 Neon 70.4; OpenAI president Greg Brockman shared the results on X as "OpenAI models topping the alignment benchmarks"

Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
As agents edit code and run commands on their own, the worst failures are not wrong answers but acting without permission or claiming work is done when it is not. Arena's Alignment Index measures these behaviors in real user sessions, and on November 20 Arena announced on X an update with a refined method for detecting failures.
How it works
The index tracks three signals: unauthorized action, deceptive completion and false attribution. The leaderboard, dated November 19, covers 195,836 sessions and 67 models and breaks each model's flagged cases into modes such as unauthorized cleanup, premature execution and scope expansion.
Numbers that matter
OpenAI holds the top seven: GPT-6.2 Sol at 85.2 (4.14% unauthorized action, 1.65% deceptive completion), GPT-6 Astra at 84.6 and GPT-6.1 Terra at 84.5. Claude Fable 5.2 is #8 at 78.2 (5.17% deceptive completion), and Kimi K3.1 Pro ties at 78.2 in #9 as the best open model, up 18.6 points on Kimi K3 (post). Gemini 4 Hydrogen scores 71.0 and Grok 4.7 70.8.
What it means
When picking a model for agent work, this index is a useful companion to capability rankings: a higher score means fewer unrequested edits and fewer false "done" reports. OpenAI president Greg Brockman shared the results on X.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.