AI Coding Live News

Google DeepMind and Artificial Analysis Launch MKA, a Multimodal Knowledge Agent Benchmark of 200 Real Visual Tasks: Claude Fable 5.2 Leads at 57%, Gemini 4 Carbon Scores 54% at About One-Fifth the Cost per Task

On November 10, Google DeepMind and Artificial Analysis launched MKA (Multimodal Knowledge Agent), a benchmark that tests whether agents can read visually dense files such as architectural drawings, engineering schematics, and clinical scans and deliver slide decks, spreadsheets, reports, and marked-up images. Its 200 private tasks span 10 industries and 83 occupations, with about 70 expert rubric checks per task. Artificial Analysis tested 21 models at launch: Claude Fable 5.2 (max) leads with a Climb WSR of 57%, followed by Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%. Fable averages $31.97 per task versus $6.29 for Carbon. Ten public tasks are on Hugging Face.

Google DeepMind and Artificial Analysis Launch MKA, a Multimodal Knowledge Agent Benchmark of 200 Real Visual Tasks: Claude Fable 5.2 Leads at 57%, Gemini 4 Carbon Scores 54% at About One-Fifth the Cost per Task

Key Takeaways

  • Scale: 200 private tasks across 10 industries and 83 occupations, each equal to 3–8 hours of work for a seasoned professional, graded on about 70 yes/no checks weighted 75% factual accuracy and 25% visual design (Google blog)
  • Leaderboard (Climb WSR): Claude Fable 5.2 (max) 57%, Gemini 4 Carbon (high) 54%, Claude Opus 5.5 (max) 53%, out of 21 models tested (AA leaderboard)
  • Cost: Fable 5.2 averages $31.97 per task, more than 5x Carbon ($6.29) and about 29x GPT-6.1 Sol ($1.11); the cheapest is GPT-6 Luna (Max) at $0.18
  • No single winner: Fable 5.2 leads 5 verticals, Opus 5.5 leads Education and Construction, Gemini 4 Argon leads Science, and Carbon leads Healthcare; Word document and text file outputs are hardest, with no model above 52%
  • Method: every model runs in the open-source Stirrup agent harness (up to 250 turns) and is graded by a non-Google model, GPT-6 Luna, as an agentic judge that agreed with domain experts 96% of the time; 10 public tasks are on Hugging Face under Apache-2.0
Read full story

Anthropic Publishes an Official Prompting Guide for Claude Fable 5.2: Saying "This Codebase Is Yours" Lifts Duplicate-Code Merging From Under a Tenth of Tries to About Half, and Old Prompts Need a Cleanup

Anthropic has published "Prompting Claude Fable 5.2" in the Claude developer docs, covering where Fable 5.2 behaves differently from earlier models. On November 10, Simon Willison highlighted one tip: tell the model in the system prompt that "this codebase is yours," and it becomes more willing to merge duplicate code and fix related problems. The guide also offers text that stops the model from ending turns before the work is done, ways to keep users informed during long agentic turns, advice on limiting subagents, and a warning to clean up instructions written for Fable 5.1 and Opus 5.5. All figures in the guide come from Anthropic's own testing (vendor figures).

Anthropic Publishes an Official Prompting Guide for Claude Fable 5.2: Saying "This Codebase Is Yours" Lifts Duplicate-Code Merging From Under a Tenth of Tries to About Half, and Old Prompts Need a Cleanup

Key Takeaways

  • Code ownership: on small coding tasks, adding "This codebase is yours" made the model merge duplicate code in about half of tries vs under a tenth without it, but output tokens rose to about 1.5x and roughly 1 in 6 to 10 tries changed how an existing feature behaves
  • Finishing the task: replaying agentic coding sessions at medium effort, the official "Finishing work" text cut sessions that ended with doable work left by about half, with about 30% more output tokens
  • Progress updates: Fable 5.2 returns mid-turn updates as thinking blocks whose text is empty at the default display of "omitted"; a harness reminder that the user has not heard from the model roughly halved the longest silent stretch
  • Migration cleanup: lines such as "answer directly without deliberating" no longer stop thinking at medium and above, so lower effort instead; harnesses that edit earlier turns invalidate thinking blocks and get a 400 error
  • Effort: the API default is high, where most of the capability gain already appears; use xhigh only where evals show a gain, and max for ceiling tests or open-ended math and science problems
Read full story

Artificial Analysis: Claude Fable 5.2 Takes #1 on the AA-3D Interactive 3D Benchmark at 1271 Elo, 67 Points Clear, and Leads the VIALS-AA Scientific Image Reasoning Benchmark at 67%

Independent evaluator Artificial Analysis published two new results on November 10. On AA-3D, which tests how well a model acting as an agent builds interactive three.js web scenes, Claude Fable 5.2 (Max) scores 1271 Elo, 67 points ahead of second-place Claude Sonnet 5.5 (1204); its 95% confidence interval of 1246 to 1297 does not overlap any other model, so the lead is outside the margin of error. It is 132 points above Fable 5.1 (1139) and ahead of the top OpenAI and Google entries, GPT-6 Astra (1168) and Gemini 4 Argon (1147). On VIALS-AA, a life-science image reasoning benchmark, Fable 5.2 scores 67%, ahead of previous leader Gemini 4 Carbon (64%) and GPT-6 Astra (63%). AA also found opposite strategies: Fable averages about 37 tool calls per task, Astra about 8.

Artificial Analysis: Claude Fable 5.2 Takes #1 on the AA-3D Interactive 3D Benchmark at 1271 Elo, 67 Points Clear, and Leads the VIALS-AA Scientific Image Reasoning Benchmark at 67%

Key Takeaways

  • AA-3D: Fable 5.2 (Max) 1271 Elo vs Sonnet 5.5 1204 and Opus 5.5 1202; its 95% CI of 1246-1297 overlaps no other model
  • Generational gain: 132 points above Fable 5.1 (1139); the top OpenAI entry GPT-6 Astra scores 1168, the top Google entry Gemini 4 Argon 1147
  • AA-3D setup: each model works as an agent in an offline sandbox for up to 3 hours / 250 turns with Blender, Rapier physics, Python and a headless browser; human panelists vote on blind pairs after at least 30 seconds per scene
  • VIALS-AA: Fable 5.2 67%, Gemini 4 Carbon 64%, GPT-6 Astra 63%
  • Opposite tool strategies: Fable makes about 37 tool calls per task and 1.60 per turn (highest on the board); Astra makes about 8, the fewest AA has measured
Read full story

First Third-Party Scores for Claude Fable 5.2: Cursor Says It Tops CursorBench at 60% on Max Effort and 56.9% at Default (Fable 5.1: 49.2%); Perplexity Reports a Record 0.851 on WANDR at $14.51 per Task

Within an hour of Anthropic releasing Claude Fable 5.2, Cursor and Perplexity both shipped the model and posted results from their in-house evals. Cursor's official account says Fable 5.2 scores 56.9% on CursorBench at default (High) effort, up from 49.2% for Fable 5.1, and 60% at max effort, the top score on the board. Cursor's chart puts the default setting at about $9.70 average cost per task. Perplexity's official account says Fable 5.2 set a record 0.851 on its WANDR benchmark at $14.51 per task, on the performance-cost Pareto frontier; Gemini 4 Argon scores 0.802 ($7.88), GPT-6 Astra 0.735 and Fable 5.1 0.684. Both are vendor-run evals, not independent public benchmarks.

First Third-Party Scores for Claude Fable 5.2: Cursor Says It Tops CursorBench at 60% on Max Effort and 56.9% at Default (Fable 5.1: 49.2%); Perplexity Reports a Record 0.851 on WANDR at $14.51 per Task

Key Takeaways

  • CursorBench (Cursor's own eval): Fable 5.2 scores 56.9% at default effort vs 49.2% for Fable 5.1, a gain of about 7.7 points; max effort reaches 60%, the top of the board
  • Cost (per Cursor's chart): default High effort averages about $9.70 per task; max effort costs far more, roughly 3x the default in the chart
  • WANDR (Perplexity's own eval): Fable 5.2 hits a record 0.851 at $14.51 per task; Gemini 4 Argon 0.802 / $7.88, GPT-6 Astra 0.735 / $13.34, Fable 5.1 0.684 / $13.16
  • Availability: selectable in Cursor now; in Perplexity Computer for Pro and Max subscribers, and in Search for Max only
  • Caveat: both are vendor-built evals whose task sets and methods are not fully public, so treat them as indicative
Read full story
ADSponsored

Anthropic Releases Claude Fable 5.2 and Mythos 5.2: Price Cut to $8/$40 per Million Tokens, a Record 49.1% on FrontierCode 2.0 Among Released Models, and 1,000 Agents Formalizing Fermat's Last Theorem in 30 Hours

At 18:58 UTC on November 9 (2:58 a.m. on November 10 SGT), the official Claude account released Claude Fable 5.2 and Claude Mythos 5.2. They are the same underlying model: Fable ships with cybersecurity and biology safeguards and is open to all paid plans, the Claude Platform and the major clouds, while Mythos drops some of those safeguards and is offered only to vetted organizations through the Cyber and Life Sciences Verification Programs. Pricing falls from Fable 5.1's $10/$50 to $8/$40 per million input/output tokens, output is up to 30% faster, and an optional fast mode generates up to 2.5x faster for $16/$80. By Anthropic's numbers, Fable 5.2 at Xhigh effort scores 49.1% on FrontierCode 2.0, ahead of Opus 5.5 (45.5%) and GPT-6 Astra (42.8%); at High effort it matches Opus 5.5's best for about half the cost per task. The API model name is claude-fable-5-2.

Anthropic Releases Claude Fable 5.2 and Mythos 5.2: Price Cut to $8/$40 per Million Tokens, a Record 49.1% on FrontierCode 2.0 Among Released Models, and 1,000 Agents Formalizing Fermat's Last Theorem in 30 Hours

Key Takeaways

  • Pricing: $8 / $40 per million input/output tokens, 20% cheaper than Fable 5.1's $10 / $50; fast mode generates up to 2.5x faster at $16 / $80
  • Coding (Anthropic's numbers): FrontierCode 2.0 at Xhigh effort 49.1% (Opus 5.5 45.5%, GPT-6 Astra 42.8%); at High effort 45.3%, matching Opus 5.5's best for about half the cost per task; FrontierSWE v2 69.9%
  • Long-running agents: 1,000 Fable 5.2 agents formalized Fermat's Last Theorem in Lean in 30 hours, while Opus 5.5 in the same setup finished only 26% in 18 hours; one engineer had it coordinate over 1,000 subagents for 8 days to build a prototype OS that boots on a laptop
  • Other scores (Anthropic's numbers): RoboDojo Sim 64.9% (Fable 5.1 27.6%), HLE with tools 74.2%, OSWorld 2.1 89.3%, Remote Labor Index 41.3%; Mythos 5.2 CVE Rediscovery 89.6%
  • Safeguards: Fable and Mythos are the same model; Fable 5.2's classifiers block about 55% less often than 5.1's; the preserved thinking anti-distillation feature now applies to all accounts
Read full story