The official Scale AI Remote Labor Index (RLI) leaderboard shows Anthropic's Claude Fable 5.2 in first place with a 41.25% automation rate, ahead of Opus 5.5 at 21.25%, GPT 6 Astra at 20.83% and Gemini Argon 4 at 17.08%. RLI, built by the Center for AI Safety and Scale AI, uses real paid remote freelance projects to test whether agents can deliver acceptable work end to end. The result drew wide discussion on X after @kimmonismus shared it.

Key Takeaways

  • ✓Official Scale AI RLI leaderboard: Fable 5.2 first at 41.25%, Opus 5.5 21.25%, GPT 6 Astra 20.83%, Fable 5.1 17.92%, Gemini Argon 4 17.08%
  • ✓The RLI score is the automation rate: the share of projects where the agent's deliverable meets a client-acceptable standard; the leaderboard notes there is no confidence interval for it
  • ✓Official scoring uses a private test set of 230 projects drawn from game development, product design, architecture, data analysis and video animation, together representing over 6,000 hours of real work valued at over $140,000
  • ✓The RLI site says the automation rate has doubled roughly every 3.3 months over the last 3 months; early agents on the board such as Manus 1.0 and gpt-5.2 sit around 2.5%
  • ✓At 41.25%, most projects still fail to meet the bar, so human review before delivery remains necessary
🧭

Turn your technical choice into a development budget

Compare 40 dev plans & simulate token costs vs $20/mo subscriptions

🔬

In-Depth Technical Analysis

Background

Many AI benchmarks test knowledge and reasoning questions, which says little about how much value a model creates in real work. The Remote Labor Index (RLI), built by the Center for AI Safety (CAIS) and Scale AI, takes a different approach: it uses real paid remote freelance projects and checks whether an agent can deliver client-acceptable work end to end. On the official Scale AI leaderboard, Anthropic's Claude Fable 5.2 now ranks first at 41.25%. After @kimmonismus shared the result on X it drew over a thousand likes, and the discussion was still going on November 16.

How it works

The RLI score is the automation rate, meaning the share of projects whose deliverable meets an acceptable standard; the leaderboard notes there is no confidence interval for it. Projects span game development, product design, architecture, data analysis and video animation. Costs and completion times come from the professionals who actually did the work, adding up to over 6,000 hours valued at over $140,000. Official leaderboard scoring uses a private test set of 230 projects to reduce contamination.

Numbers that matter

Official Scale AI leaderboard: Fable 5.2 41.25%, Opus 5.5 21.25%, GPT 6 Astra 20.83%, Fable 5.1 17.92%, Gemini Argon 4 17.08%, Fable 5 15.83%, and newly added DeepSeek V4.1 Flash at 9.58%. Early agents further down the board, such as Manus 1.0 and gpt-5.2, sit around 2.5%. The RLI site says the automation rate has doubled roughly every 3.3 months over the last 3 months.

Getting started

A 41.25% rate means top agents can already finish a real share of freelance projects on their own, but close to six in ten projects still miss the bar. The benchmark hands agents a complete brief; in real work, writing the brief and approving the result still falls to people. Teams that want to try Fable 5.2 on design, data analysis or small development jobs should start with well-scoped pilot tasks and keep a human review step.

Action HubReady to adopt this in production?

Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.

Up NextAnthropic Launches Claude Meetings Beta: Claude Joins Google Meet Calls as a Visible Participant, Builds Docs and Whiteboards Live and Writes the Notes, Rolling Out to Team and Enterprise
Read Now→