Tooling9d ago
Cloudflare discloses a cross-request KV cache leak in GLM-5.3 on Workers AI: a config mismatch plus an SGLang bug let 155 responses carry other users' data over 13 days
In an Oct 26 incident report on its blog, Cloudflare said that between Aug 27 and Sep 9, 155 responses from GLM-5.3 on Workers AI (@cf/zai-org/glm-5.3) to prompts longer than 681,280 tokens contained data from other customers' recent requests, all returned with HTTP 200. The cause was a prefill KV cache pool smaller than the 1,048,576-token context window, combined with an SGLang bug in disaggregated prefill/decode mode. Cloudflare found no evidence of deliberate exploitation, has fixed it, and recommends operators upgrade to SGLang v0.5.22 or later.
Release9d ago
Baidu open-sources Unlimited OCR 2: SR-SWA keeps memory and latency nearly flat on very long documents, scoring 77.1 on MarathonDocBench per Baidu
On Oct 27, Baidu's official @Baidu_Inc account announced the open-source release of Unlimited OCR 2 on Hugging Face under the MIT license. Its new SR-SWA attention compresses text-visual interactions into a fixed-size reference set, which Baidu says lets it parse documents of any length within a finite context window. Baidu reports 84.6 on MPDocBench (dozens of pages) and 77.1 on MarathonDocBench (hundreds of pages), both self-reported.
Tooling15d ago
Eagle releases EAGLE-4, a speculative-decoding drafter that adapts to batch size and hardware; it reports 961.6 tok/s on Qwen3.8-27B for a single request
On Oct 20, inference company Eagle released EAGLE-4, the fourth generation of the EAGLE speculative-decoding family. One drafter resizes its speculative tree for the batch size, hardware and workload while keeping the target model's output distribution. Eagle reports 961.6 tokens/s for one request on Qwen3.8-27B with 4x B300. The code ships as eagle-harness v0.1.0 with plugins for SGLang, vLLM and an experimental TokenSpeed backend. Many reposts call it open source, but the official license is source-available, not OSI open source.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.