AI Coding Live News

Together AI Launches Together Compiler: Agents Continually Optimize Production Inference, Up to 2.6x Higher TPS

On November 4 Together AI launched Together Compiler (TKCC), which pairs a swarm of AI agents with a persistent inference engine to keep rewriting inference kernels for real traffic, models and hardware, as part of what it calls Polymorphic Inference. The company says that within 24 hours it reached up to 2.6x higher TPS on models such as Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash, up to 50% faster on customer workloads, and it already serves trillions of tokens a day in production.

Key Takeaways

  • Together Compiler recasts inference optimization from a classic compiler problem into an agentic search: agents propose kernel changes, a persistent engine checks speed and correctness, and only verified changes reach production
  • Together says that within 24 hours it reached up to 2.6x higher TPS on Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash, and up to 50% faster on customer workloads
  • On InferenceX low-latency batch size 1 it reports 1,111 TPS on MiniMax M3.1-Flash and 1,205 TPS on DeepSeek v4.1-Flash (company-reported); SemiAnalysis said it validated one of the results
  • The persistent engine hot-reloads candidate forward passes, cutting time to serving to 9 seconds versus 467 seconds for restarting an engine, with 252x higher experiment throughput than baseline agentic optimization systems
  • It already serves trillions of tokens a day in Together's production stack, where 40+ models are now optimized by agents
Read full story

Ricursive Open-Sources Ripple-65B Preview: A 65B Dense Model Pretrained From Scratch, TerminalBench 2.1 80.5, Apache 2.0

Ricursive Intelligence released its first model, Ripple-65B Preview, on November 4 (18:24 UTC), open-sourcing the instruction-tuned model (BF16 and NVFP4) and a base model on Hugging Face under Apache 2.0. The official model card describes a 65B dense Transformer pretrained on 12T tokens with up to 1M context, and self-reports TerminalBench 2.1 80.5, SWE-bench Pro v2.0 Hard 79.1 and DeepSWE 1.1 44.3, ahead of much larger US open-weight models on agentic coding. It fits on local hardware such as the DGX Spark. CEO Anna Goldie says it cost about a tenth of the prior comparable state of the art.

Ricursive Open-Sources Ripple-65B Preview: A 65B Dense Model Pretrained From Scratch, TerminalBench 2.1 80.5, Apache 2.0

Key Takeaways

  • Ripple-65B Preview is a 65B dense model with a thin-and-deep shape (80 layers, d_model 5120), moving from an Olmo 3 baseline to a Gemma 3N-like architecture.
  • It was pretrained on 12T tokens at 8K sequence length, then went through mid-training, long-context training, SFT and mixed-domain RL, with up to 1M context.
  • Self-reported TerminalBench 2.1 is 80.5, above Reflection Beam 501B (80.1) and Muse Spark Open 816B (76.4), but SWE-bench Verified 76.5 and DeepSWE 1.1 44.3 show it does not lead everywhere.
  • Ricursive says the core team grew from one to five people and code compute efficiency improved 16x versus the Olmo 3 ladder; it reaches about 250 tok/s on 2x B200.
  • Hugging Face hosts BF16, NVFP4, base and DFlash2 speculative-decoding checkpoints, all under Apache 2.0.
Read full story
Linked Models & Agents:
⚡ Ripple-65B Preview⚡ Reflection Beam 501B⚡ Muse Spark Open⚡ Nemotron 3 Ultra🌐 Ricursive Intelligence🌐 NVIDIA

OpenAI Shares Progress on ChatGPT and Codex for Hardware Engineering: GPT-6 Astra Designs a Codex Device in Under a Week, Mazda Joins for Aerodynamics Research

On November 3, OpenAI published "Accelerating engineering for the physical world," describing how its models help with mechanical design, electrical design and simulation. Two employees used GPT-6 Astra to design a device for talking to Codex (enclosure, PCB, firmware and software) in under a week, and a contract manufacturer built a small prototype batch in under a month, though manufacturing readiness still took several rounds of human review. Token usage across OpenAI hardware teams has grown 23-fold since February. On BenchCAD 2.0, a newer internal model averaged 0.875 versus 0.594 for GPT-6 Astra at its highest reasoning setting (OpenAI-reported). OpenAI is also expanding its work with Chip Ganassi Racing and announced a new research collaboration with Mazda.

OpenAI Shares Progress on ChatGPT and Codex for Hardware Engineering: GPT-6 Astra Designs a Codex Device in Under a Week, Mazda Joins for Aerodynamics Research

Key Takeaways

  • In a test with no direct human edits, GPT-6 Astra handled the look, enclosure structure, component selection, PCB layout, UI and firmware; two employees finished the design in under a week as a side project, and prototypes came back in under a month, per the OpenAI blog.
  • OpenAI names the gaps: Astra did not consistently account for the manufacturer's production limits, tolerances or assembly details, so manufacturing readiness took several rounds of review.
  • OpenAI hardware and infrastructure teams use Codex and ChatGPT for CAD models, 2D manufacturing drawings, simulation and FEA setup, and design reviews; token usage has grown 23-fold since February (OpenAI-reported).
  • BenchCAD 2.0 (reconstructing 3D parts and assemblies, identifying circuit-board connections): a newer internal model scored 0.875 versus 0.594 for GPT-6 Astra at its highest reasoning setting (OpenAI-reported; the internal model is unnamed and unreleased).
  • Collaborators: Chip Ganassi Racing is testing 3D-scan-to-editable-CAD and 2D drawing workflows; Mazda is working with OpenAI researchers on airflow and structural simulations for vehicle design, with results promised "in the coming months."
Read full story

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

On Nov 2, semiconductor and AI research firm SemiAnalysis released GDPx, a benchmark of agentic capability across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (RSI), hardware design, materials science, alignment and software engineering. Its 206 tasks were built by 14 specialist data partners, with SemiAnalysis doing final QA. On the official leaderboard Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%; nearly every model scores 0 on finance and legal. SemiAnalysis says safety classifiers drag down Opus 5.5 and GPT-6 Astra, and plans a new version every 3–6 months. All scores are as reported by SemiAnalysis.

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

Key Takeaways

  • Scale: 11 domains, 14 data partners and 206 tasks, with a default time limit of 8 hours per task (GDPx site)
  • Leaderboard (as reported by SemiAnalysis): Claude Opus 5.5 12.0%, Gemini 4 Carbon 10.9%, GPT-6 Astra 9.6%, Muse Wonder Full 9.0%, Claude Fable 5.1 8.5%, GPT 6.1 Sol 8.4%, Claude Sonnet 5.5 8.2%
  • Lower tier and open weights: DeepSeek 4.1 Flash 1017 5.4%, GPT-6 Luna 5.1%, Grok 4.7 3.9%, GLM 5.3 2.7%; almost every model scores 0 on finance and legal
  • Method: each model runs in its native harness where one exists (Claude Code for Opus, Codex for Astra, Antigravity for Carbon), otherwise OpenCode; the headline is pass@1 over three attempts with all domains weighted equally (SemiAnalysis blog)
  • By domain: Opus 5.5 wins quant trading, hardware and chip design; Gemini 4 Carbon wins biology and software engineering; GPT-6 Astra wins RSI and materials science; Muse Wonder Full is the strongest cyber model
Read full story
ADSponsored

X discussion: Matt Pocock sketches the ideal "language for agents" (Rust speed, Go-sized design space, strictest verification) and names Bend; Chris Lattner replies with Mojo

At 17:23 UTC on 2026-10-25, TypeScript educator Matt Pocock (@mattpocockuk, author of Total TypeScript and AI Hero) posted (post) that the next great "language for agents" should be as fast as Rust, have as little design space as Go, and have the strictest type/verification system ever written, calling Bend "an interesting first effort". When we pulled it, the post had about 3,130 likes, 84 reposts, 264 replies, 786 bookmarks and 220K views, and it reached X's trending stories. Mojo creator Chris Lattner replied (reply) recommending Mojo and suggesting "can target all hardware" be added to the list. This is an opinion debate, not a verified conclusion.

Key Takeaways

  • Main post: post, published 17:23 UTC on 10-25, with about 3,130 likes, 84 reposts, 264 replies, 786 bookmarks and 220K views when we pulled it
  • Three criteria: Rust-level speed, a Go-sized design space (few ways to write the same thing), and the strictest type/verification system ever written, so the compiler rather than a human catches agent mistakes
  • Chris Lattner replied recommending Mojo, citing advanced type features such as linear types and symbolic constraints, and suggested adding "can target all hardware" (reply, about 40 likes when pulled)
  • Bend creator Victor Taelin (@VictorTaelin) also joined the thread, saying Bend blocking an agent from breaking an app is "working as intended"
  • No benchmarks or controlled comparisons were shared; whether Bend or Mojo suits agent coding remains personal opinion
Read full story
Linked Models & Agents:

Google Cloud Open-Sources TorchTPU: Native PyTorch on Google TPU Hardware with Day-Zero vLLM and SGLang Support

Google Cloud officially open-sourced TorchTPU, enabling native, zero-compilation-barrier PyTorch execution directly on Google Cloud TPU hardware (TPU v5p and TPU v6e). Overcoming the legacy friction of PyTorch/XLA's rigid graph compilation and dynamic-shape stalls, TorchTPU supports native PyTorch eager execution and custom Triton/C++ kernels. Leading high-throughput serving engines vLLM and SGLang announced immediate Day-0 support. Concurrently, Google Cloud introduced OpenAI models and Codex into its Vertex Model Garden, becoming the first hyperscaler to offer all three top frontier closed models.

Google Cloud Open-Sources TorchTPU: Native PyTorch on Google TPU Hardware with Day-Zero vLLM and SGLang Support

Key Takeaways

  • Google Cloud open-sources TorchTPU, delivering native, compilation-free PyTorch execution on TPU v5p and TPU v6e hardware
  • Eliminates PyTorch/XLA graph-compilation overhead, natively supporting PyTorch eager mode and custom Triton/C++ kernels
  • vLLM and SGLang announce immediate Day-0 TorchTPU releases, unlocking substantial throughput boosts on TPU infrastructure
  • OpenAI frontier models and Codex join Google Cloud's Model Garden, solidifying Google Cloud as the premier multi-model hyperscaler
Read full story

NVIDIA Unveils OpenShell Reference Architecture: Hardware-Isolated Security Guardrails for Autonomous Agents

At GTC Berlin, NVIDIA unveiled the OpenShell Reference Architecture alongside the Agent Knowledge Foundation within its Agent Toolkit. Addressing critical enterprise vulnerabilities where autonomous tool-using agents succumb to prompt injections, hallucinated destructive commands, or unauthorized data exfiltration, OpenShell establishes hardware-isolated runtime boundaries. Operating through hardware roots of trust, OpenShell dynamically intercepts out-of-boundary tool calls and file access in real time with sub-microsecond latency, establishing an authoritative blueprint for secure, auditable enterprise agent deployments.

NVIDIA Unveils OpenShell Reference Architecture: Hardware-Isolated Security Guardrails for Autonomous Agents

Key Takeaways

  • NVIDIA officially launched the OpenShell Reference Architecture at GTC Berlin, establishing hardware-isolated boundaries for autonomous agents
  • Implements dynamic runtime interception, strictly preventing out-of-boundary tool invocations, malicious script execution, and data leaks
  • Natively integrates with the NVIDIA Agent Toolkit and Agent Knowledge Foundation for end-to-end access control and compliance auditing
  • Solves critical enterprise security roadblocks preventing production rollout of high-privilege coding and autonomous operations agents
Read full story
Linked Models & Agents:

NVIDIA launches OpenShell Reference Architecture at GTC Berlin: Vera Rubin, Vera CPUs and BlueField-4 Sentry for hardware-enforced safe AI agents

At GTC Berlin on Oct 21, NVIDIA announced the OpenShell Reference Architecture: building on the open-source OpenShell secure runtime, it pairs Vera Rubin for inference, Vera CPUs for agent harnesses, and BlueField-4 DPUs running NVIDIA Sentry for in-silicon policy enforcement, so enterprises can build, contain, observe and govern agents on-prem and at the edge. NVIDIA says it is available now; partners named include Anthropic, OpenAI, Meta, Microsoft and ServiceNow. The software is Apache 2.0.

NVIDIA launches OpenShell Reference Architecture at GTC Berlin: Vera Rubin, Vera CPUs and BlueField-4 Sentry for hardware-enforced safe AI agents

Key Takeaways

  • Full stack: Vera Rubin for model inference, Vera CPUs for agent harnesses/tool use, BlueField-4 + Sentry for in-silicon policy and quarantine (vendor claims)
  • Vendor claims: Vera Rubin NVL72 about 30x throughput per megawatt on agentic workloads; Sentry can quarantine a boundary-breaking agent in milliseconds and boost agent capacity by up to 20%
  • Runtime: deny-by-default, enforcement outside the agent process; sandboxes have no direct network, syscalls are filtered in-kernel, credentials are injected only onto allowed requests
  • Stated agent paths include Claude Code, Codex, GitHub Copilot CLI, OpenCode, OpenClaw, LangChain Deep Agents; deploy with Docker, Podman or Kubernetes via Helm
  • Start: curl install script then openshell sandbox create; docs at docs.nvidia.com/openshell; GitHub NVIDIA/OpenShell (Apache 2.0) and PyPI package openshell
Read full story

EdgeKernelBench: Unified Benchmark and LEKA Agent for LLM-Driven Kernel Generation on Heterogeneous Edge Backends

Researchers introduced EdgeKernelBench and the LEKA agent architecture in arXiv:2610.24648, establishing the first unified framework for evaluating and generating low-level high-performance kernels across heterogeneous edge backends. While LLM-driven kernel generation has demonstrated success on datacenter GPUs, edge deployments remain plagued by hardware diversity, fragmented compiler toolchains (CUDA, OpenCL, TileLang, Triton), and strict memory-compute constraints. EdgeKernelBench bridges this gap through a unified intermediate representation (IR) suite spanning 331 canonical workloads. The companion Layered Edge Kernel Agent (LEKA) hierarchically decomposes optimization decisions, unlocking efficient automated kernel synthesis on edge silicon.

EdgeKernelBench: Unified Benchmark and LEKA Agent for LLM-Driven Kernel Generation on Heterogeneous Edge Backends

Key Takeaways

  • EdgeKernelBench released in arXiv:2610.24648, establishing the premier unified evaluation harness for edge kernel generation
  • Spans CPU/LLVM, GPU/CUDA, and GPU/OpenCL stacks with 331 canonical compute patterns across 650 concrete benchmark workloads
  • Introduces LEKA (Layered Edge Kernel Agent), hierarchically decoupling compilation and schedule decisions across IR abstraction tiers
  • Solves single-shot generation failures (where models achieved only 25.5% [email protected]), paving the way for automated edge NPU acceleration
Read full story
Linked Models & Agents:

vLLM Narrows Gap with DeepInfer to 6% on NVIDIA GB200, Reaching 1M Tokens/s per GPU on GB300

Hours after DeepSeek open-sourced DeepInfer, the vLLM project team released official benchmark parity data serving DeepSeek-V4.1-Flash. Utilizing DeepSeek's identical decoding setup (131K context window, nextn=3 speculative decoding), vLLM closed its throughput performance delta to within 6% of DeepInfer across batch sizes 8 through 256 on NVIDIA GB200 systems. On GB300 hardware, vLLM attained up to 1,000,000 tokens/second per GPU and 826 tokens/second per user, generating $38.96 in hourly GPU revenue at DeepSeek off-peak rates. vLLM simultaneously shipped day-0 runtime support for FastH3-Omni and Spark-X2.6-4B.

vLLM Narrows Gap with DeepInfer to 6% on NVIDIA GB200, Reaching 1M Tokens/s per GPU on GB300

Key Takeaways

  • vLLM published direct benchmark comparisons, demonstrating that its serving throughput on NVIDIA GB200 is within 6% of DeepInfer
  • Achieves unprecedented scaling on NVIDIA GB300: up to 1,000,000 tokens/second per GPU and 826 tokens/second per active user stream
  • Generates up to $38.96 per GPU-hour in revenue when mapped to DeepSeek's production off-peak API pricing schedule
  • Delivers immediate Day-0 support for the FastH3-Omni video model and the 1M-token Apache 2.0 model Spark-X2.6-4B
Read full story
Linked Models & Agents:
ADSponsored

Eagle releases EAGLE-4, a speculative-decoding drafter that adapts to batch size and hardware; it reports 961.6 tok/s on Qwen3.8-27B for a single request

On Oct 20, inference company Eagle released EAGLE-4, the fourth generation of the EAGLE speculative-decoding family. One drafter resizes its speculative tree for the batch size, hardware and workload while keeping the target model's output distribution. Eagle reports 961.6 tokens/s for one request on Qwen3.8-27B with 4x B300. The code ships as eagle-harness v0.1.0 with plugins for SGLang, vLLM and an experimental TokenSpeed backend. Many reposts call it open source, but the official license is source-available, not OSI open source.

Eagle releases EAGLE-4, a speculative-decoding drafter that adapts to batch size and hardware; it reports 961.6 tok/s on Qwen3.8-27B for a single request

Key Takeaways

  • Vendor figure: 961.6 tokens/s on Qwen3.8-27B with TokenSpeed, BF16, 4x B300, TP4, one request, T=0 (paper Figure 1(a))
  • Repo measurements: vLLM on 4x B300 goes from 241.6 to 722.8 tok/s at concurrency 1 (2.99x) and 1.70x at concurrency 8; SGLang gets 1.91x at concurrency 8 and 1.35x at 64
  • AMD numbers too: SGLang on one MI355X gets 2.07x at concurrency 8 and 1.56x at 32 (ROCm 7.2)
  • Design: sliding-window draft layer, low-rank candidate head and append-only draft cache; the harness calibrates verify latency offline and grows the tree while gain per cost stays high
  • License: Eagle 4 Source-Available; free for noncommercial use, commercial use only under $20M trailing 12-month revenue and outside competing offerings
Read full story

SemiAnalysis Discloses Google & Inferact TPUv7 Submission: TorchTPU vLLM Megakernels Drive 2.8T Kimi K3, Surpassing GB200 NVL72 and B300 in High-Interactivity TCO

SemiAnalysis revealed the first Google TPUv7 benchmark submission for the 2.8T Kimi K3 MoE model, led end-to-end by Inferact (the creators of vLLM) and Google using the TorchTPU vLLM stack. At a hardware rental cost of just $1.21/chip-hour (vs $2.31 for GB300 and $1.86 for GB200), TPUv7 delivers ~2.2M tokens per $1 TCO in the ultra-high interactivity regime (~390 tok/s/user), outpacing NVIDIA GB200 NVL72, B300, and AMD MI355X on cost-performance.

SemiAnalysis Discloses Google & Inferact TPUv7 Submission: TorchTPU vLLM Megakernels Drive 2.8T Kimi K3, Surpassing GB200 NVL72 and B300 in High-Interactivity TCO

Key Takeaways

  • Inferact (creators of vLLM) and Google submitted their first preliminary TPUv7 benchmark for the 2.8T Kimi K3 MoE model
  • TPUv7 chip rental sits at $1.21/hour, significantly below GB300 ($2.31), B300 ($2.26), and GB200 ($1.86)
  • Delivers ~2.2M tokens per $1 TCO at ~390 tok/s/user, beating NVIDIA GB200 NVL72, B300, and AMD MI355X in the ultra-high interactivity regime
  • Built with fused TorchTPU vLLM megakernels and the TPU-Sync KVCache offload library for production PD disaggregation
Read full story
Linked Models & Agents:

Anthropic gives a six-person METR team employee-like access and no veto over publishing its risk assessments

On Oct 15 Anthropic said it has onboarded six METR employees with company hardware, office space and access to internal tools and systems roughly on par with its own risk assessment teams. METR will publish its analysis of Anthropic's risk levels, incidents and practices; Anthropic cannot veto publication and may only require redactions in predefined categories. Anthropic also outlined a three-tier access model for evaluators and updates on SecureBio, Accenture and Faculty.

Anthropic gives a six-person METR team employee-like access and no veto over publishing its risk assessments

Key Takeaways

  • Six METR staff get company hardware, office space and internal access roughly on par with Anthropic's risk assessment teams (per Anthropic)
  • Anthropic cannot veto METR publications; it may only require redactions in predefined categories such as customer data, privileged legal material and model architecture
  • Evaluator access has three tiers: tier 1 publishes with approval, tier 2 publishes without approval, tier 3 (METR) adds employee-like access
  • SecureBio is reviewing the chem/bio chapter of the August Risk Report; partnerships with Accenture and Faculty cover red-teaming and alignment assessments, with early talks including Hugging Face
  • The announcement post drew about 537K views and 3.8K likes within roughly three days
Read full story

AMD introduces Helion agentic AI rack blueprint: OCP-based design, up to 32,768 CPU cores for hyperscale, MI350P all-in-one for enterprise, partner configs due November 2026

On 2026-10-12 AMD introduced Helion, an OCP-based open rack blueprint for agentic AI infrastructure building on Helios. Hyperscale CPU-only racks with 6th Gen EPYC can reach up to 32,768 cores for agent execution tied to separate inference; enterprise all-in-one racks combine EPYC, Instinct MI350P and general-purpose compute. Open hardware/software plus an integrated token router for frontier and open-source model calls; AMD plans to contribute the spec to OCP, with partner configurations targeted for November 2026.

Key Takeaways

  • Official: AMD Newsroom and GlobeNewswire (2026-10-12)
  • Configs: CPU-only 6th Gen EPYC up to 32,768 cores (hyperscale agent execution) vs all-in-one EPYC + Instinct MI350P + general compute (enterprise)
  • Stack: OCP blueprint, air/liquid cooling, open software, integrated token router across frontier and open-source models
  • Go-to-market: spec to be contributed to OCP; Cisco/Dell/Hon Hai/HPE/Lenovo/QCT/Supermicro/Wiwynn quoted; partner configs planned November 2026 (vendor)
  • Vs Helios: Helios is training/large-scale inference rackscale; Helion is the agent-execution-flexible blueprint — do not treat as one SKU
Read full story

LoGRA: Scaling LLM Reinforcement Learning via Low-Rank Gradient Sketches, Slashing Memory by 45.7% for 27B Models

Reinforcement learning (RL) post-training is foundational to scaling reasoning capabilities in large language models. However, its memory footprint remains a prohibitive bottleneck: storing full-sized gradient buffers alongside multi-state Adam optimizer moments frequently triggers out-of-memory (OOM) failures when attempting to train large models (such as 27B parameters) on standard compute hardware. In newly released research (arXiv:2610.06647), researchers introduce LoGRA, an efficient RL post-training framework available within the open-source Molt library (github.com/skzhang1/labs-molt). LoGRA compresses training representations by retaining useful learning signals inside compact low-rank gradient sketches accumulated directly during backpropagation, facilitating both memory-efficient parameter updates and streamlined policy synchronization. To counteract instability from gradient compression, LoGRA introduces a Predicted-KL step control mechanism that forecasts policy change magnitudes prior to parameter updates, dynamically attenuating disruptive updates. Evaluated across reasoning benchmarks, LoGRA reduces average training memory by up to 45.7% with zero degradation in task performance, successfully sustaining stable RL training of a 27B model for over 1,100 steps on a single eight-GPU node where standard dense Adam crashes with OOM.

LoGRA: Scaling LLM Reinforcement Learning via Low-Rank Gradient Sketches, Slashing Memory by 45.7% for 27B Models

Key Takeaways

  • LoGRA framework released, introducing low-rank gradient sketches to slash LLM RL training memory by up to 45.7%
  • Breaks single-node 8-GPU memory barriers, sustaining stable RL training of a 27B model for over 1,100 steps
  • Predicted-KL step control prevents policy drift, achieving full parity with standard AdamW on reasoning benchmarks
Read full story