On 2026-11-05 Boson AI (the team of Mu Li and Alex Smola) released a research preview of Higgs Realtime Duplex: a 45B MoE front model that slices voice, video, and tool results into 160 ms chunks so it can listen, watch, speak, and render its own face at once, while thinking, calling tools, or delegating hard questions to a larger backend model without pausing. Vendor-reported scores include Big Bench Audio 98.9, tau-Voice 69.8, and Full-Duplex-Bench v1.5 75.5. The API is waitlist-only for now; the earlier Higgs Realtime is available today.
Key Takeaways
- ✓Shape: a 45B MoE front model, further trained on about 2T tokens of text, audio, and video; every stream is cut into 160 ms chunks, silence included
- ✓Background work: it thinks and calls tools asynchronously while talking, and delegates hard tasks with full context to a larger backend model (a frontier model is recommended)
- ✓Vendor-reported scores: Big Bench Audio 98.9 (top of the table), IFEval 94.1, tau-Voice 69.8 (GPT-Live-1 about 74.5), Full-Duplex-Bench v1.5 75.5 (GPT-Live-1 80.1, TML 77.8)
- ✓Cost design: audio and video use about 2.6K tokens per minute, about 2.6K-5K with thinking and tools; the target is an hour of talk costing at most one-tenth of an hour of human time
- ✓Availability: the Duplex API is waitlist-only; voice and face swap zero-shot from a reference clip or a single image
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Core Background & Industry Pain Points
Voice agents usually work turn by turn: the user finishes, the model thinks, then it speaks, so tool calls create dead air and interruptions are handled poorly. OpenAI GPT-Live, Gemini Live, and Thinking Machines' interaction models all target real-time behavior, but most push reasoning and tool use to a large background model, which makes latency and cost hard to control. Boson AI wants one mid-sized model to talk and work at the same time.
Architecture Highlights & Internals
Higgs Realtime Duplex slices time instead of turns: user voice, camera frames, tool results, and backend replies arrive in 160 ms chunks, and at each step the model reads one chunk and writes the next chunk of speech, face latents, and background thinking. Thinking and tool calls can span chunks, and the model can start speaking from partial tool results. The front model is a 45B MoE that hands hard tasks, with full context, to a larger backend model. Post-training uses multi-teacher on-policy distillation (MOPD, including GRPO) plus a simulated self-improvement loop. A flow-matching audio decoder and a causal DiT render the voice and face, swappable zero-shot from a reference clip or a single image.
Authoritative Benchmarks & Measured Scores
All figures are vendor-reported by Boson, with comparisons taken from public TML, Artificial Analysis, Scale, and OpenAI results: Big Bench Audio 98.9 (TML Interaction Small 96.5); Audio MultiChallenge APR 44.9 (GPT Realtime-2 xHigh 48.5); Full-Duplex-Bench v1.5 75.5 (GPT-Live-1 80.1, TML 77.8); tau-Voice 69.8 (GPT-Live-1 Astra medium 74.5); IFEval 94.1. It does not lead everywhere, but a 45B model trades blows with TML's 276B model. No independent third-party reproduction yet.
Developer Hands-on Guide
Duplex is a research preview; join the API waitlist from the official blog. To integrate today, start with the existing Higgs Realtime (see docs.boson.ai and the pricing page). Good fits include support lines, spoken-language tutoring, meeting assistants, and hands-free engineering where the agent must talk while it looks things up. Before launch, test interruption handling and false triggers, and cap backend delegation cost so long sessions do not blow up token spend.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.