AI evaluation nonprofit METR published a research update on November 20 covering public models up to Opus 5.5 and GPT-6 Astra. On its expanded TH1.1 plus TH2 task suite, frontier 50% time horizons double roughly every 3.6 months (95% CI 2.1 to 4.9), and naive extrapolation reaches the suite's longest 200-hour tasks in July 2027. METR says current public evidence is insufficient to rule out drastic AI R&D acceleration in the next couple of years, and that capabilities are getting harder to measure.

Key Takeaways

  • ✓On the combined TH1.1 and TH2 suite, frontier 50% time horizons double roughly every 3.6 months (95% CI 2.1 to 4.9), close to the trend OpenAI reported on internal research tasks
  • ✓Naive extrapolation has 50% time horizons reaching the suite's longest tasks (200 hours) in July 2027
  • ✓On coding and optimization tasks, models usually match or beat the skilled humans METR recruited; on contest-style tasks with many entrants they place very high but not at the top
  • ✓The new tasks are so hard that only 9% of human time estimates come from successful human baselines; the rest use records, expert interviews and extrapolation
  • ✓METR introduces the METR Capability Index (MCI), which, like Epoch's ECI, estimates ability directly from scores; trend fits range from slight deceleration to moderate acceleration depending on methodology
🧭

Turn your technical choice into a development budget

Compare 40 dev plans & simulate token costs vs $20/mo subscriptions

🔬

In-Depth Technical Analysis

Background

METR used to argue catastrophic risk was low because models' autonomous capabilities had broad limitations, as measured by time horizons. In an @METR_Evals thread on X it says that argument is now harder to make, so it extended its task suite.

How it works

METR added more cheaply verifiable tasks plus tasks that are hard to verify or require judgment, and tested public models up to Opus 5.5 and GPT-6 Astra. It also introduces the METR Capability Index (MCI), which, like Epoch's ECI, estimates task difficulty and agent ability directly from scores without human time estimates.

Numbers that matter

Per the METR post, frontier 50% time horizons double roughly every 3.6 months (95% CI 2.1 to 4.9), and naive extrapolation reaches the suite's 200-hour tasks in July 2027. Only 9% of human time estimates come from successful baselines. Agents improve even on hard-to-verify tasks but still show worse judgment than human experts there.

What it means for developers

Tasks with clear, fast feedback such as coding and optimization are increasingly safe to hand to agents, while work without fast feedback still needs human review. METR also notes continued reward hacking and growing eval awareness in frontier models.

Action HubReady to adopt this in production?

Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.

Up NextIBM Releases LLMON, an LLM-Native Markup Language, With Granite 4.1 3B and 8B Preview Models: Reserved Tokens Separate Instructions From Data and Block 47% More Indirect Injection Attacks on OPI Than the Best Frontier Model Tested
Read Now→