AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

AI Benchmarks Track@aibenchmarks·Collected 48d ago

👀58

🔥94

Android Bench 2.0 Released: Shifting Evaluation to Multi-Day Long-Horizon Tasks

Researchers released Android Bench 2.0, moving benchmarks from single-function bug fixes to complex multi-day Long-Horizon Tasks (LHT). The suite evaluates coding agents on building apps from scratch, complex multi-library version migrations, and cross-platform verification.

Key Takeaways

Deprecates trivial unit bug benchmarks in favor of multi-day full-system app synthesis
Tests real-world architectural scaffolding, breaking library upgrades, and UI regression
Top frontier agents currently achieve only 31.4% success, exposing long-horizon bottlenecks

Read full story

Linked Models & Agents:

⚡ Claude 5.1⚡ GPT-6 Astra 🤖 Cursor 🤖 Claude Code

#Android Bench 2.0#Long Horizon Tasks#Coding Agents#Benchmarks#Software Engineering

Epoch AI@EpochAIResearch·Collected 48d ago

👀59

🔥90

View on 𝕏 ↗

Epoch AI launches Benchmark Reviews to audit 15 AI benchmarks

Epoch AI launched Benchmark Reviews to audit the quality of AI benchmarks themselves. In the first set of 15 benchmarks, four were marked Verified, nine Flawed, and two Not Enough Info, giving developers a stronger basis for interpreting model and agent leaderboards.

Key Takeaways

The initial review covers 15 AI benchmarks: four Verified, nine Flawed, and two Not Enough Info.
Reviews apply to specific benchmark versions and publish methodology and limitations.
Benchmark-quality audits can reduce leaderboard-driven errors in model selection, coding evaluation, and agent assessment.

Read full story

Linked Models & Agents:

#Epoch AI#Benchmark Reviews#AI benchmarks#Evaluation#LLM

Epoch AI@EpochAIResearch·Collected 66d ago

👀38

🔥86

View on 𝕏 ↗

Epoch AI Benchmarks GPT-5.6 Sol on Slay the Spire: Strong Tactical Play but Strategic Bottlenecks Remain

Epoch AI evaluated GPT-5.6 Sol on Slay the Spire via computer-use plugins, finding that while the model reliably commands the UI and clears low ascensions (A4-A9), long-term strategic decision-making remains a key hurdle.

Key Takeaways

Computer-use vision execution is reliable, successfully identifying complex UI elements and state transitions;
Strategic planning latency is high, requiring 5-8 hours of thinking per run with occasional tactical misplays;
Epoch initiated follow-up benchmarks on Slay the Spire 2 with live streamed autonomous agent runs on Twitch.

Read full story

Linked Models & Agents:

⚡ GPT-5.6 Sol