Benchmark5d ago
Epoch AI: open-weight GLM 5.3 autonomously exploited the patch gap in six Firefox releases for about $1,000 each
Epoch AI reported on Oct 29 that, in a deliberately basic setup emulating an unsophisticated attacker, Z.ai's open-weight GLM 5.3 read security fixes that were public but not yet shipped and autonomously built exploit chains for all six Firefox releases from Sep 17 to Oct 26, letting a single webpage visit read local files. Marginal cost was about $1,000 per version and the fastest run took 12 hours. Mozilla patched a previously undisclosed bug the model also used in Firefox 159.0 on Oct 27.
Tooling2d ago
X discussion: open-source AI builder Jun Song says he turned down an anonymous black hat group offering huge money to post-train open-weight models on exploit data
At 02:20 UTC on Nov 1, open-source AI builder Jun Song (@jun_song, who builds Supergemma) posted on X that an anonymous black hat group offered him an "astronomical" sum to post-train open-weight models with reinforcement learning on the exploit data it holds. He said the volume was so large he would have believed it was an undercover FBI dataset, that he turned it down, and that what scares him is that someone else will eventually take the deal. As of 17:10 SGT the post had about 2,029 likes, 222K views and 771 bookmarks, and it has been rewritten as a parody template. This is a personal account; the offer, the data volume and the group cannot be verified, so it is treated here as a discussion.
Benchmark65d ago
Together: GLM-5.3 Flash cascade hits 80.9% on DeepSWE at $1.70/task
Together ran about 900 DeepSWE rollouts of GLM-5.3 vs GLM-5.3 Flash: 69.0% pass@1 at ~$3.99 per task versus 63.4% at ~$0.24, a 17x cost gap. Distillation mostly removed first-try reliability, not coverage—the pass@4 gap shrinks to 2.6 points. A Flash-then-escalate cascade reached 80.9% at about $1.70 per task, beating the full model alone at less than half the price. Coding agents should default to Flash and promote on test failure, but Flash broke an already-passing baseline in 6.9% of rollouts versus 4.4%, so diffs still need a regression gate.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.