On November 18 the official Tencent Hy account (@TencentHunyuan) announced open weights for Hy4 flash, releasing an FP8 quantized version and the pre-trained Base model alongside it on Hugging Face, ModelScope and GitCode under the Tencent Hy License. According to the model card it is a mixture-of-experts model with 220B total parameters (including 64B Engram parameters) and about 12B active per token, takes text and image input natively, and supports up to 1M tokens of context. vLLM and SGLang ship official images, and a GGUF build is available for testing.

Key Takeaways

  • ✓Three releases: Hy4 flash (instruct), Hy4 flash-FP8 (quantized) and Hy4 flash-base (pre-trained), all downloadable
  • ✓Scale: 220B total parameters (64B of them Engram), about 12B active per token, 1M (1,048,576) token context
  • ✓Architecture: sparse and sliding-window attention mixed 1:3, 16 of 512 routed experts active per token, plus a built-in 0.85B DFly2 speculative-decoding drafter
  • ✓Deployment: official vLLM image vllm/vllm-openai:hy4-flash and an SGLang image; the reference setup runs the FP8 model with 8-way tensor parallelism
  • ✓Licensed under the Tencent Hy License (not Apache or MIT), so read the terms before commercial use
Tencent Hy Open-Sources Hy4 flash Weights, Including FP8 and Base Versions: 220B Total, About 12B Active, 1M Context, Day-0 vLLM and SGLang Support
🖼️Official Media
Click to view high-res
⚡

Key Decision Metrics at a Glance

TL;DR Verdict
🧭

Not enough VRAM? Compare cloud API and self-hosting costs

Compare 40 dev plans & simulate token costs vs $20/mo subscriptions

🔬

In-Depth Technical Analysis

Background

Hy4 flash launched as an API on November 16, pitched as a low-cost model for everyday office and agent work. Many teams want to run this kind of fast, cheap model on their own hardware to keep data in-house and avoid per-token billing, and the open-weight release covers that.

How it works

Per the model card, Hy4 flash has a 42-layer backbone (1 dense layer plus 41 MoE layers), alternating sparse attention and 512-token sliding-window attention at a 1:3 ratio. Routed experts run in a lower-dimensional latent space, with 16 of 512 experts selected per token, and learned bigram and trigram Engram embeddings are added. The vision encoder is adapted from SigLIP2, and a 0.85B DFly2 drafter handles long-context speculative decoding.

Benchmarks

In Tencent's base-model comparison, Hy4 flash Base scores 88.7 on MMLU, 71.1 on MMLU-Pro and 57.8 on SuperGPQA. On Tencent's internal WorkIndex, Hy4 flash scores 42.1 versus 39.7 for Hy4 preview, at about 1/13 of the cost per task. These are Tencent's own numbers with no third-party reproduction yet.

What it means for developers

You can stand up an OpenAI-compatible server with the official vLLM or SGLang images; the reference setup runs the FP8 model on 8 GPUs. Transformers main-branch support is still being merged. If memory is tight, try the GGUF build from AngelSlim first. The license is the Tencent Hy License, so check the terms before commercial use.

Action HubReady to adopt this in production?

Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.

Up NextThinking Machines Opens Inkling 2 Small Weights on Hugging Face Under Apache 2.0: 594B-Total / 17B-Active MoE, Native NVFP4 Runs on Two B300s, Day-0 Support from SGLang, vLLM, Cloudflare and More
Read Now→