LMSYS Launches the SGLang Consortium at the First SGLang Summit: NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk Join as Founding Members to Help Shape the Open-Source Inference Engine Roadmap

Background and the problem

AI compute is shifting from training to inference. Citing Deloitte, LMSYS says inference will be about two-thirds of all AI compute in 2026, up from about half in 2025. How many tokens a given set of GPUs produces, and at what cost, depends heavily on the inference engine. SGLang is the open-source, full-stack inference engine maintained by LMSYS and part of the PyTorch ecosystem; LMSYS says it runs on hundreds of thousands of GPUs and is used in production by Meta, Microsoft, Google, Amazon, Thinking Machines Lab, Cloudflare and others. The more companies depend on it, the more their needs for new models and new hardware risk being handled in isolation, which calls for a formal channel.

Architecture and how it works

What launched is a collaboration structure, not new code. Each of the eight founding members names one technical representative and joins two roadmap meetings a year, contributing engineering time, deployment feedback, testing resources or joint development; working groups can collaborate between meetings, with public updates on roadmap direction and progress. The four focus areas are frontier model support, reasoning and multimodal workloads, distributed inference, and cross-hardware optimization. The key constraint: SGLang maintainers keep authority over technical decisions and code acceptance, so members cannot force merges, and the project stays open to everyone. With NVIDIA, AMD and Intel all at the table, cross-hardware support will be discussed on one shared roadmap.

Benchmarks and measured results

The announcement includes no new benchmarks or performance numbers. The only figures are official scale claims: SGLang runs on hundreds of thousands of GPUs and powers some of the largest production inference deployments in the world. RadixArk, Modal and the official SGLang account also posted the same day confirming they joined as founding members, and Modal said SGLang helped it serve trillions of tokens. Performance updates are more likely to come from summit sessions on NVIDIA Vera Rubin, AMD Instinct and other platforms.

Getting started for developers

SGLang users do not need to change anything for now: the license, repository and contribution process stay the same. The consortium will publish roadmap updates, so the LMSYS blog and the SGLang repository are the places to watch for priorities on day-0 model support, multimodal and distributed inference. Teams deploying on AMD or Intel hardware can expect more official optimization work. The summit runs November 12-13 at Fort Mason in San Francisco, with a main-stage livestream covering reinforcement learning, multimodal models and hardware platforms.