Artificial Analysis Launches VIALS-AA: Handshake Life-Sciences Visual Reasoning Board; Vendor Lead Gemini 4 Carbon ~64%

Core Background & Industry Pain Points

Life-sciences workflows hinge on reading unpolished experimental images—gels, flow-cytometry plots, plasmid maps, phylogenies—and unstable interpretation can mislead decisions. Handshake AI VIALS covers 161 expert tasks across eight domains; on 2026-11-06 Artificial Analysis launched VIALS-AA, an independent harnessed rerun that scores reliable success across three repeats rather than a single lucky pass.

Architecture Highlights & Internals

Runs use the open Stirrup harness: models receive the question and image paths, with offline Python code execution plus an image-viewing tool. Each task is repeated three times; GPT-6 Luna (medium) grades answers with upstream judge prompts. The headline metric is pass^3 (correct on all three repeats), alongside Average of 3 and pass@3. The article also breaks tool calls into viewing, preparation, and measurement, showing similar scores can come from different strategies.

Authoritative Benchmarks & Measured Scores

Figures below are Artificial Analysis / vendor-reported, not independently reproduced here. Leaders: Gemini 4 Carbon (high) ~64% (103/161 all-three-correct), GPT-6 Astra (max) ~63%, Claude Opus 5.5 (max with fallback) ~61%. GPT-6.1 Sol ~59% at ~$0.056/task versus Opus ~$1.64/task. Open weights: Kimi K3.1 Pro ~45%. Kimi solves ~119/161 at least once but only ~73 on all three repeats—consistency separates the frontier.

Developer Hands-on Guide

Start at artificialanalysis.ai/articles/vials-aa and the leaderboard page; inspect trajectories for tool-call volume. Rank by pass^3 and Cost per Task together, not pass@3 alone. Prefer consistency for science use cases, and label vendor figures versus unreproduced local runs.