Benchmark5d ago
Artificial Analysis releases AA-Omniscience v1.1: 3,000 refreshed questions and a calibrated 300-question public set, with Gemini 4 Carbon leading at a Net Score of 54
On October 31, independent evaluator Artificial Analysis released AA-Omniscience v1.1, its knowledge and hallucination benchmark and part of Intelligence Index v5. The question set shrinks from 6,000 to 3,000, dropping previously public and saturated questions, fixing errors and ambiguities, and adding new ones, and a new 300-question public set on Hugging Face is calibrated to the private set's difficulty, with 0.99 Pearson and Spearman correlation on Net Score. Gemini 4 Carbon (high) leads with a Net Score of 54, ahead of Claude Opus 5.5 at 48 and Claude Fable 5.1 at 46; Fable 5.1 has the highest accuracy (68%) and Gemini 4 Argon the lowest hallucination rate (4%). Models score about 1.9 points lower than on v1 on average, and MiniMax M3.1 Flash drops 22 points. All figures are as reported by Artificial Analysis.
Tooling58d ago
vLLM: day-0 MiniCPM5-2B on stable — 131K context, Think/No-Think, tool calling
vLLM (@vllm_project) announced day-0 support for OpenBMB MiniCPM5-2B on stable vLLM: 2.6B dense with native 131K context, Think/No-Think from one checkpoint, and tool calling via vLLM’s minicpm5 parser. Built on stock LlamaForCausalLM with training data opened alongside weights.
Benchmark44d ago
Artificial Analysis: Grok 4.7 + Grok Build hits Coding Agent Index 56 (4th); Intelligence Index 46 puts SpaceXAI in top 4 labs
@ArtificialAnlys evaluated Grok 4.7 (xhigh): Intelligence Index 46 (+2 vs 4.6), putting SpaceXAI in the top 4 labs. With Grok Build it scores 56 on the Coding Agent Index (+9 vs 4.6), 4th among native harnesses behind Claude Fable 5.1, GPT-6 Astra, and Claude Opus 5. Pricing stays $2/$6 per 1M in/out ($0.50 cache hits), 500k context.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.