Benchmark19h ago
SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%
On Nov 2, semiconductor and AI research firm SemiAnalysis released GDPx, a benchmark of agentic capability across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (RSI), hardware design, materials science, alignment and software engineering. Its 206 tasks were built by 14 specialist data partners, with SemiAnalysis doing final QA. On the official leaderboard Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%; nearly every model scores 0 on finance and legal. SemiAnalysis says safety classifiers drag down Opus 5.5 and GPT-6 Astra, and plans a new version every 3–6 months. All scores are as reported by SemiAnalysis.
Benchmark1d ago
Memdoor reproduces GVS5H: a team of five Qwen3.8-27B agents averages 9.8/10 on the first 10 LiveCodeBench Hard problems, against 8.0 for a single Fable 5 call
On Oct 31 the open-source coding agent Memdoor (GitHub guregodevo/memdoor, Apache 2.0) published its team-solve workflow: using the GVS5H paper authors' exact prompts and loop, five Qwen3.8-27B instances act as planner, two ideators, solver and manager, talk only through files on disk, and a script that runs the tests decides success. Over five passes on the first 10 LiveCodeBench Hard problems from the GVS5H list it averaged 9.8/10, against 9.6 for the authors' own harness and 8.0 for a single Fable 5 call; 49 of 50 runs solved the problem for $35.79 in total, and every request and reply is public. The Hacker News post has about 200 points. This is a community reproduction on only 10 problems, and the author notes that training-data contamination cannot be ruled out.
Benchmark1d ago
X discussion: is GPT-6 Astra robotics' "GPT-3 moment"? Ex-OpenAI researcher Will Depue says specialized ML got "bitter lessoned"
Over the past day, X has been debating whether GPT-6 Astra can work directly as a robot policy. Former OpenAI researcher Will Depue (whose bio lists Sora 1 and 2 and o3/4o post-training) called Astra beating specialized robotics models "deeply surprising", then described it as "the final death of specialized ML". Investor Freda Duan posted that "Astra IS robotics' GPT-3 moment", drawing about 84K views. Third-party evaluations point both ways: in a RoboDojo paper, Astra averages 22.48% success and ranks above all 40 public policies, while an EBench report puts it at 46.7%, third of nine models. This is a discussion of opinions, not an official OpenAI conclusion.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.