👀192
🔥84
llama.cpp v0.6.0 ships GLM-5.3-Flash (320B) and Qwen4Exp MTP speculative decoding, a new llama_batch_ext API, and up to ~3x faster few-row mat-mul on Apple GPUs
ggml-org released llama.cpp v0.6.0 on Oct 5 (v0.5.0 was Sep 23). It brings GLM-5.3-Flash (GLM5-Next, a 320B text+vision hybrid MoE) into a stable tag, adds MTP speculative decoding for Qwen4Exp (~1.5x decode on DGX Spark), introduces the llama_batch_ext / llama_process() API for mixed token and embedding batches, adds few-row MMA mat-mul kernels on Metal (up to ~3x for speculative and batched decoding), and bumps ggml to v0.26.0. Session file formats are bumped too.
Key Takeaways
- New models: GLM-5.3-Flash (GLM5-Next, 320B KDA/DSA hybrid text+vision MoE), Clef decision model, Ling 3.0 VL, LFM2.5-Encoder
- Speed: Qwen4Exp MTP speculative decoding ~1.5x decode on DGX Spark; Metal few-row MMA mat-mul up to ~3x
- API: llama_batch_ext + llama_process() for mixed token/embedding batches; server and examples migrated
- Compat: LLAMA_SESSION_VERSION 11 and LLAMA_STATE_SEQ_VERSION 4, so old session/state files must be regenerated
- Server: /v1/models reports modalities, /v1/embeddings takes typed vision/audio/video input, new /v1/systemone endpoint
Linked Models & Agents:
