At AI Engineer NYC on 2026-10-13, Revolut AI ops lead David Abitbol said ~2,000 engineers now default to self-hosted GLM-5.3 on ~56 NVIDIA B200 GPUs at a blended ~$0.30 per million tokens—about one-tenth prior Anthropic/OpenAI invoices. Speaker charts claim ~60% better coding time and ~70% lead time. Claude Code/Codex remain hard-case exceptions; vision is routed elsewhere.
Key Takeaways
Source: Revolut AI ops lead David Abitbol, AI Engineer NYC 2026 talk (2026-10-13) + official transcript; Register coverage from 2026-11-03 (speaker figures)
Scale: ~2,000 engineers default to self-hosted GLM-5.3 on ~56 NVIDIA B200 GPUs; speaker blended cost ~$0.30 per million tokens
Cost: invoice about one-tenth prior Anthropic/OpenAI spend; ~60% coding-time and ~70% lead-time improvements claimed (not isolated from other changes)
Routing: GLM for text/code; vision to Gemini/GPT-class models; Claude Code/Codex kept for hard exceptions
Ops: same GPU count tuned TTFT ~14.3s→1.7s and +76% RPS; sandbox approvals + telemetry stressed for bank compliance
On November 5 Meta AI (@AIatMeta) said Muse Realtime Avatar, previewed at Connect, is rolling out in the US Muse app, turning Muse Realtime Voice into interactive live avatars. The research blog describes an audio-driven Diffusion Transformer that shares the speech-token stream with voice for lip-sync and expression; a single GB200 session streams 448×768 at 25 fps with about 870 ms from end of user turn to first synchronized audio-video byte, with Meta Video Seal invisible watermarking. Muse is for users 18+; no global schedule or developer API was announced.
Zoom
Key Takeaways
US Muse app rollout of Realtime Avatar after the Connect preview (@AIatMeta)
Shared speech VQ tokens drive audio decode and avatar video; causal chunks with self-forcing distillation cut ~120 teacher NFEs to 2 student NFEs (~60x)
Vendor perf: 448×768@25fps, ~870ms first-byte latency; ~8x serving capacity vs BF16 baseline, up to 12 concurrent sessions on one GB200
Vendor preference study favored Muse over Runway Characters and HeyGen LiveAvatar on overall and most axes
Safety: Video Seal invisible watermark; blog demos are not the full in-app avatar set; no public developer API stated
On November 3, model-compression company Pruna AI open-sourced P-H3-Image, which turns MiniMax-H3, the open-weights video model MiniMax released this summer, into a text-to-image and image-editing model. Per Pruna: sampling drops from 50 to 8 steps, a 4 MP image on one B200 goes from 12.4 s to 2.6 s, the 4-bit set is a 20 GB download (vs 143 GB for H3), and peak VRAM at 8 MP is about 21 GB. It renders 2 to 16 MP and edits with multiple reference images. Weights are on Hugging Face with native ComfyUI support. It keeps the MiniMax H3 Community License, which by default excludes the EU, UK, South Korea and the US.
Zoom
Key Takeaways
Speed (Pruna figures): 8 steps instead of 50; on one B200 a 4 MP image goes from 12.4 s to 2.6 s and 8 MP from 30.6 s to 5.8 s; the 4-bit set does 4 MP in about 6.3 s on an L40S
Size: the transformer shrinks from 33B to 20B parameters and the text encoder is distilled from 32B to 4B (67 GB to 9 GB); downloads are 55 GB in bf16 and 20 GB in 4-bit vs 143 GB for H3
Memory: peak VRAM at 8 MP is about 56 GB in bf16 and 21 GB in 4-bit; the bf16 diffusers pipeline with the prompt upsampler needs about 100 GB
Capabilities: renders 2 to 16 MP, strongest at photorealistic and documentary photography, film stills, anime and graphics with short text; supports multi-reference editing (8 images per the X post, 9 per the blog)
License: MiniMax H3 Community License; the EU, UK, South Korea and US need a separate license from MiniMax, and products must show "Powered by MiniMax H3"
Indian AI company Sarvam released Sarvam Prism on November 3, a "System One" decision model for enterprise workflows: given a context and questions with fixed answer options, it returns a probability for every option in a single forward pass instead of generating text. Sarvam says that across the 25 decision models it tested, Prism averages 71.8 over three indices with a median response time of 44 ms on one B200, the highest score under 100 ms, and scores 82.5 on its Indian-language benchmark. Weights are on Hugging Face under Apache 2.0, with a hosted API as well. All scores are Sarvam's own measurements.
Zoom
Key Takeaways
Trained on the Sarvam 105B (105B-10B MoE) base; it only outputs option probabilities, so the answer is always one of the given options.
Vendor-reported indices: Jev Decision Index 67.0, Sarvam's recreation of the Artificial Analysis decision index 65.9, internal Indian-language benchmark 82.5, average 71.8; Perplexity Decider v1.2 averages 68.0 at about 117 ms.
Calibration: on English answers at 90%+ confidence it expects 3.3% errors and makes 2.2%; in Indian languages 5.4% expected vs 4.4% actual, while rivals underestimate errors by 1.5x to 4.8x.
By sorting options into a fixed order before scoring, it changed 0 of 2,000 answers across four shuffled option orders.
On 700 recorded agent runs it judges correctly 78.5% of the time overall, vs 70.7% for Claude Haiku 5.5.
Self-hosting needs one FP8-capable GPU with about 120 GB free memory (such as B200 or H200) and about 106 GB of disk for weights.
a16z's Charts of the Week for Oct 30 (by Moses Sternstein) looks at the "Rack Spread": GPU rental prices are rising while token prices fall. The official a16z X account summed it up on Oct 31: since May, average token prices are down 67% and token spend is up more than 120%. By the article's napkin math, Ramp data gives roughly 122%, and Ornn's blended price multiplied by OpenRouter volume gives roughly 160%. A separate post says that since mid-July B300 rentals are up 106%, B200 up 66% and H100 up 17%. a16z also stresses that these indexes come from small, biased samples and do not show true total demand.
Zoom
Key Takeaways
Prices and spend: since May, average token prices are down 67% and token spend is up more than 120% (a16z post on X, a16z figures)
Two napkin estimates: Ramp token-spend data shows about +122%; Ornn blended price times OpenRouter volume shows about +160% (Charts of the Week, described by the author as napkin math, not a rigorous method)
GPU rentals since mid-July: B300 +106%, B200 +66%, H100 +17%, while average token prices fell 58% (a16z post)
AI spend per employee: an Atlanta Fed survey puts the mean above $1,000, while Ramp's median is $15.70 a month ($188 a year)
Reach: the two posts have about 371 and 492 likes and 38K and 96K views (as of 18:15 SGT)
On 2026-10-23 (16:53 UTC) the official @blocks account unveiled Dessa, and Jack Dorsey called it the most authentic full-duplex speech model ever built. The official page block.xyz/dessa says Dessa is a true speech-to-speech model trained only on recordings of human conversation, with no text and no pretrained models. It is full-duplex, so it listens and talks at the same time, cuts in and backchannels naturally, runs faster than real time, and runs locally on a MacBook Pro M4 Max or better. Block says it was trained on a single NVIDIA GB200 NVL72 and learned language from hundreds of times fewer words than LLMs. There is no public API or weights yet; a small group of AI safety researchers is getting early access.
Zoom
Key Takeaways
Trained only on human conversation audio, from scratch, with no text and no pretrained models (official page)
Full-duplex speech-to-speech: listens while talking, cuts in and backchannels, and natively hears and expresses emotion, sarcasm and laughter
Inference runs faster than real time and runs locally on a MacBook Pro M4 Max or better
Block says training used a single NVIDIA GB200 NVL72 and needed hundreds of times fewer words than LLMs
No public API, weights or third-party benchmarks yet; early access is limited to a small group of AI safety researchers
Hours after DeepSeek open-sourced DeepInfer, the vLLM project team released official benchmark parity data serving DeepSeek-V4.1-Flash. Utilizing DeepSeek's identical decoding setup (131K context window, nextn=3 speculative decoding), vLLM closed its throughput performance delta to within 6% of DeepInfer across batch sizes 8 through 256 on NVIDIA GB200 systems. On GB300 hardware, vLLM attained up to 1,000,000 tokens/second per GPU and 826 tokens/second per user, generating $38.96 in hourly GPU revenue at DeepSeek off-peak rates. vLLM simultaneously shipped day-0 runtime support for FastH3-Omni and Spark-X2.6-4B.
Zoom
Key Takeaways
vLLM published direct benchmark comparisons, demonstrating that its serving throughput on NVIDIA GB200 is within 6% of DeepInfer
Achieves unprecedented scaling on NVIDIA GB300: up to 1,000,000 tokens/second per GPU and 826 tokens/second per active user stream
Generates up to $38.96 per GPU-hour in revenue when mapped to DeepSeek's production off-peak API pricing schedule
Delivers immediate Day-0 support for the FastH3-Omni video model and the 1M-token Apache 2.0 model Spark-X2.6-4B
On Oct 20, the FastVideo team at UCSD's Hao AI Lab open-sourced FastH3 Omni, a distilled version of MiniMax H3's reference-to-video-and-audio (Ref2VA) model that generates video with synchronized stereo audio from text plus up to nine images, video clips and voice references. Sampling drops from 50 steps to eight transformer evaluations, cutting end-to-end latency 5.6-11.9x on eight GB200s by the team's measurements. Training code is open, with NVFP4 weights for DGX Spark and RTX 5090 and ComfyUI support.
Zoom
Key Takeaways
Steps: eight transformer evaluations versus 50 for base MiniMax H3, via PDD plus DMD distillation
Speed (team figures): 5.6-11.9x lower end-to-end latency across 27 matched workloads on 8x GB200; a 15-second single-image request drops from 139.69s to 14.14s
Inputs: up to nine images, three videos and three audio clips, at most 12 references; output is 24 FPS video with 32 kHz stereo audio
Local: NVFP4 QAD weights for DGX Spark (128 GB) and RTX 5090 (32 GB), 480p by default; FP8 weights for RTX 4090
License: MiniMax-H3 Community License, not Apache or MIT
Meta's engineering blog on October 19 gave the first detailed look at Prometheus, the New Albany, Ohio supercluster that trained the Muse family, including Muse Wonder. Phase one is complete with hundreds of thousands of GPUs, and the full build targets about 1 million. Muse Wonder training reached more than 85% goodput in pretraining and more than 90% in reinforcement learning, with a mean time between interruptions of about 47K GPU hours, since raised above 54K by firmware fixes. It runs GB200 NVL72 and GB300 NVL72 platforms, with Vera Rubin NVL72 arriving soon.
Zoom
Key Takeaways
Scale: phase one trained the Muse family; it has hundreds of thousands of GPUs today, targets about 1 million, and runs across buildings as a single machine.
Reliability: Muse Wonder hit 85%+ pretraining goodput and 90%+ for RL; MTBI was about 47K GPU hours and is now above 54K.
Interruptions: a mid-2025 peak of nearly 1,000 events per 8K GPUs per day was cut by 99% over a year of engineering.
Sparing: each NVL72 rack runs a 64-GPU slice with 8 in-rack spares, plus about 2% spare racks; a 32,768-GPU job needs 523 racks or 37,656 GPUs, about 13% extra.
Network: each building fabric links up to 20,736 GPUs, and the aggregation network spans 2 km to over 100 km.
Viggle released JST-2 on October 19. Instead of learning in pixel or video space, it represents the world as persistent "atoms" in 3D space (Physics-Native Tokens) and uses one architecture to generate scenes, characters and motion. Viggle says a 3D scene takes about one second on an NVIDIA B200 at under $0.01 of inference, with up to 64 input views kept as spatial context, and characters animate without a separate rigging step. It is free to try in the PINOC browser app and is also available via API and MCP.
Zoom
Key Takeaways
Scene generation: about 1 second per scene (NVIDIA B200, excluding network latency) at under $0.01 inference cost; Viggle cites about 5–6 minutes for Marble (vendor figures).
Spatial context: up to 64 input views, expandable one view at a time in real time, output as 3D Gaussian splats rendered locally in the browser.
Character blind tests (run by Viggle): Average Score 58.4% vs Meshy 7.1 and 73.2% vs Tripo H3.1; face quality 63.0% and 82.0% (50% means parity).
Motion blind tests (run by Viggle): text-to-motion 74.8% vs Cartwheel Swing and 80.0% vs HY-Motion-1.0; video-to-motion 89.1% vs DeepMotion Animate 3D.
Access: free in the PINOC browser app, plus API and MCP for agents such as Codex and Claude Code; JST-3 aims to replace rule-based collisions and object interactions with learned physics.
Traditional speculative decoding relies on static target hidden features that fail to supply varying contextual needs across draft positions. HKU researchers introduced ReaDy (arXiv:2610.20503). Designed specifically for Hybrid Linear Attention (HLA) architectures combining Transformers with recurrent linear attention states, ReaDy leverages compact recurrent states as dynamic context for drafting. On a single NVIDIA B200 GPU serving Qwen3.8-27B, ReaDy achieves an acceptance length of 4.53 and 302.78 TPS, accelerating inference by 2.4x over standard autoregressive decoding.
Zoom
Key Takeaways
HKU introduced ReaDy (arXiv:2610.20503), the first speculative decoding framework customized for Hybrid Linear Attention (HLA) LLMs
Reuses compact recurrent states as dynamic context for drafting, overcoming the static context bottleneck of feature-based drafters
Achieves 302.78 TPS for Qwen3.8-27B on a single NVIDIA B200 GPU (2.40x over AR and +7.97% over SOTA DSpark)
Reaches an acceptance length of 4.53 tokens using an ultra-lightweight Rank-8 reader
The SGLang team released v0.5.22 featuring 950 PRs from 245 contributors: DeepSeek-V4.1 delivers up to 2.3x more throughput per GPU on NVIDIA H200 and 1.9x on AMD MI350X/MI355X via DSpark. The new InstantTensor loader cuts model loading time by 10x. B200 decodes Qwen3.8-Flash-Next 12% faster, while B300 speeds up GLM-5.3-Flash by 10% and yields 1.85x decode throughput for MiMo-V2.6-Pro. SGLang also adds native Apple Silicon PyTorch support and ComfyUI integration.
Zoom
Key Takeaways
SGLang v0.5.22 released with 950 PRs from 245 contributors across heterogeneous platforms
DeepSeek-V4.1 achieves up to 2.3x throughput per GPU on NVIDIA H200 and 1.9x on AMD MI350X/MI355X via DSpark
New InstantTensor loader speeds up large model checkpoint ingestion by 10x
Qwen3.8-Flash-Next decodes 12% faster on B200; MiMo-V2.6-Pro reaches 1.85x decode throughput on B300
PD disaggregation cuts p95 TTFT in half (-50%) on long prompts, alongside native Apple Silicon and ComfyUI diffusion support
SemiAnalysis revealed the first Google TPUv7 benchmark submission for the 2.8T Kimi K3 MoE model, led end-to-end by Inferact (the creators of vLLM) and Google using the TorchTPU vLLM stack. At a hardware rental cost of just $1.21/chip-hour (vs $2.31 for GB300 and $1.86 for GB200), TPUv7 delivers ~2.2M tokens per $1 TCO in the ultra-high interactivity regime (~390 tok/s/user), outpacing NVIDIA GB200 NVL72, B300, and AMD MI355X on cost-performance.
Zoom
Key Takeaways
Inferact (creators of vLLM) and Google submitted their first preliminary TPUv7 benchmark for the 2.8T Kimi K3 MoE model
TPUv7 chip rental sits at $1.21/hour, significantly below GB300 ($2.31), B300 ($2.26), and GB200 ($1.86)
Delivers ~2.2M tokens per $1 TCO at ~390 tok/s/user, beating NVIDIA GB200 NVL72, B300, and AMD MI355X in the ultra-high interactivity regime
Built with fused TorchTPU vLLM megakernels and the TPU-Sync KVCache offload library for production PD disaggregation
On 2026-10-15 Microsoft AI published a technical post and PDF saying it has finished pre-training its in-house base model MAI-Base-2: MoE in every layer (1 shared + 7-of-512 routed experts), 80B active / 2.8T total, 102 layers, native encoder-free vision, a 50T-token mixture and a March 2026 knowledge cutoff. Per MAI, it beats every open-weight model tested on held-out BPB and its recipe is 4.2–12.3x more pre-training-FLOPs-efficient than DeepSeek-V4-Pro-Base. It is still in post-training: no API, no weights and no release date yet; Microsoft plans to license weights to trusted partners via Foundry.
Zoom
Key Takeaways
Scale: 80B active / 2.8T total, 102 layers, hidden 8704, MoE in every layer (1 shared + 7 of 512 routed); MAI-Base-1 was 35B/1T, 78 layers, alternating dense/MoE (PDF)
Compute: 16K GB200s for 38 days (Aug–Sep 2026), 21% average MFU (vs 2.5 PFLOPS BF16), 89% goodput; the Muon step is 2.1% (165 ms) of iteration time
Efficiency (vendor estimate via scaling-law fits): 4.2–12.3x fewer FLOPs than DeepSeek-V4-Pro-Base to match its BPB; 6.3–12.7x better than the June MAI-Base-1 recipe
Data: 50T-token mixture, 18% vision-language (4.5T unique), ~36% more unique tokens than Base-1, described as organic data with no distillation
Availability: still in post-training, no API/weights/date; weights to be licensed to trusted partners through Foundry; MAI-Base-3 prep underway (blog)
On Oct 9 the vLLM team, with Inferact, Red Hat and NVIDIA, shipped early Vera Rubin NVL72 support: CUDA 13.4 nightly images already serve DeepSeek, Kimi, GLM and MiniMax. Vendor-reported results: up to 7.84x per-GPU throughput vs GB200 on SemiAnalysis AgentX (MiniMax M3, matched interactivity) and up to 3.7x vs GB300 NVL72 on the MLPerf Inference v6.1 VLM benchmark (Qwen3-VL-235B-A22B).
