NVIDIA Research has released the Nemotron-Labs-Sigma family on Hugging Face: 14 checkpoints at 3B and 8B, covering base, instruct, few-step distilled and self-speculative decoding variants. Instead of generating one token at a time, the models denoise Gaussian noise in a 16-dimensional continuous embedding space and emit text in parallel blocks. NVIDIA Principal Scientist Morteza Mardani announced the release on X on November 16. The weights use the NVIDIA Nemotron Open Model License and allow commercial use.
Key Takeaways
- ✓Scale: 14 checkpoints at 3B and 8B, including Base, Instruct, distill-nfe2/4/8/16 and distill-spec variants, all under the nvidia account on Hugging Face
- ✓Method: diffusion denoising in a 16-dimensional continuous token-embedding space, generating blocks of tokens (32 to 64 by default) in parallel while reusing committed tokens through a standard KV cache
- ✓Speed: NVIDIA research director Arash Vahdat says on X that the distilled models stay strong with just 4 sampling steps and that speculative decoding is 3.5x faster
- ✓The distill-spec models draft a 32-token block with the diffusion head and verify it with their own autoregressive head, so output matches greedy autoregressive decoding
- ✓Limits: up to 8,192 tokens of context (4,096 for the spec models), English only, NVIDIA GPU in bfloat16 required, no CPU inference

Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
Mainstream LLMs are autoregressive: they produce one token at a time, so long outputs are slow. In images, diffusion models already generate in parallel with few steps, but text is discrete, and most earlier diffusion language models used masked (discrete) diffusion. NVIDIA's Nemotron-Labs-Sigma takes another route and runs diffusion in a continuous space, the same way image diffusion works.
How it works
According to the model cards, the models denoise Gaussian noise step by step in a 16-dimensional continuous token-embedding space, with the denoiser and the embeddings trained together. Text is decoded in parallel blocks; each block goes through a number of DDPM or DDIM sampling steps, and earlier committed tokens are reused through a standard KV cache. The 8B model uses a Ministral-style decoder-only Transformer. The distill-spec variants add an autoregressive head: the diffusion head drafts 32 tokens at once and the autoregressive head verifies them, so the output matches greedy autoregressive decoding.
Numbers that matter
The model card says the 8B model's reasoning accuracy is competitive with state-of-the-art masked diffusion language models; detailed comparisons are in the paper, arXiv:2610.02665. NVIDIA research director Arash Vahdat says on X that the distilled models still do well with just 4 sampling steps and that speculative decoding is 3.5x faster. These are the team's own numbers and have not been reproduced by third parties yet.
What it means for developers
The weights are under the nvidia account on Hugging Face and load with transformers and trust_remote_code, with no custom kernels needed. You need an NVIDIA GPU running bfloat16; output on CPU is unusable. Context is capped at 8,192 tokens and only English is supported. NVIDIA recommends the DDIM sampler; fewer steps run faster at a small quality cost. It is a good fit for research on few-step generation or for trying faster draft decoding on your own GPUs.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.