AI Coding Live News

Epoch AI: open-weight GLM 5.3 autonomously exploited the patch gap in six Firefox releases for about $1,000 each

Epoch AI reported on Oct 29 that, in a deliberately basic setup emulating an unsophisticated attacker, Z.ai's open-weight GLM 5.3 read security fixes that were public but not yet shipped and autonomously built exploit chains for all six Firefox releases from Sep 17 to Oct 26, letting a single webpage visit read local files. Marginal cost was about $1,000 per version and the fastest run took 12 hours. Mozilla patched a previously undisclosed bug the model also used in Firefox 159.0 on Oct 27.

Epoch AI: open-weight GLM 5.3 autonomously exploited the patch gap in six Firefox releases for about $1,000 each

Key Takeaways

  • 6 of 6: every Firefox release from Sep 17 to Oct 26 was compromised using only information public at each release (Epoch report)
  • About $1,000 marginal cost per version, with a 48-hour budget and the fastest run at 12 hours; total Firefox research spend exceeded $100,000 including setup
  • The model also rediscovered an unpublished bug on each attempt, which Mozilla fixed in Firefox 159.0 on Oct 27
  • No refusals or safeguards triggered through first-party or third-party APIs
  • Roughly $100,000 of attempts against Chrome failed so far, which Epoch says is not strong evidence Chrome is safe
Read full story

Epoch AI launches Benchmark Reviews to audit 15 AI benchmarks

Epoch AI launched Benchmark Reviews to audit the quality of AI benchmarks themselves. In the first set of 15 benchmarks, four were marked Verified, nine Flawed, and two Not Enough Info, giving developers a stronger basis for interpreting model and agent leaderboards.

Key Takeaways

  • The initial review covers 15 AI benchmarks: four Verified, nine Flawed, and two Not Enough Info.
  • Reviews apply to specific benchmark versions and publish methodology and limitations.
  • Benchmark-quality audits can reduce leaderboard-driven errors in model selection, coding evaluation, and agent assessment.
Read full story
Linked Models & Agents:

Epoch AI Benchmarks GPT-5.6 Sol on Slay the Spire: Strong Tactical Play but Strategic Bottlenecks Remain

Epoch AI evaluated GPT-5.6 Sol on Slay the Spire via computer-use plugins, finding that while the model reliably commands the UI and clears low ascensions (A4-A9), long-term strategic decision-making remains a key hurdle.

Key Takeaways

  • Computer-use vision execution is reliable, successfully identifying complex UI elements and state transitions;
  • Strategic planning latency is high, requiring 5-8 hours of thinking per run with occasional tactical misplays;
  • Epoch initiated follow-up benchmarks on Slay the Spire 2 with live streamed autonomous agent runs on Twitch.
Read full story
Linked Models & Agents: