AICoder

AI Coding Live News

First-hand AI coding updates from 𝕏/Twitter frontier labs and researchers, distilled by AI with entity benchmarks.

Explore live on 𝕏 ↗

AI Coding Live News

L
LM Studiolmstudio·
👀82
🔥80
View on 𝕏 ↗

LM Studio Bionic 1.1.10 adds the oMLX engine, bringing RAM-plus-SSD KV caching and MLX speculative decoding to local models on Macs

LM Studio posted on X at 15:32 UTC on Oct 29 that users can now download a new oMLX engine inside LM Studio and run it with MLX models they already have. The official changelog lists it in Bionic 1.1.10, alongside speculative decoding for MLX models and an in-app terminal. oMLX is an open-source Apple Silicon inference server maintained by developer jundot (Apache 2.0, about 23,000 GitHub stars) built around continuous batching and a two-tier RAM and SSD KV cache. LM Studio has not published speed comparisons for the engine.

LM Studio Bionic 1.1.10 adds the oMLX engine, bringing RAM-plus-SSD KV caching and MLX speculative decoding to local models on Macs

Key Takeaways

  • LM Studio says the oMLX engine can now be downloaded inside the app and used with existing MLX models; the post has about 540 likes (post)
  • The Bionic 1.1.10 changelog (Oct 29) also lists speculative decoding for MLX models, a public in-app terminal and llama.cpp 2.66.0 extension packs (changelog)
  • oMLX is Apache 2.0 open source with about 23,000 GitHub stars and needs macOS 15+ on Apple Silicon M1 to M5 (GitHub)
  • Its core is a two-tier KV cache: hot blocks stay in RAM, overflow is written to SSD as safetensors, and matching prefixes are restored from disk even after a restart
  • LM Studio has published no speed comparison with its existing MLX engine; one user reported that restart-to-update failed after launch and LM Studio is investigating
Read full story
Linked Models & Agents:
⚡ MLX🤖 LM Studio🤖 oMLX🌐 LM Studio🌐 Hugging Face🌐 Apple
#LM Studio#oMLX#MLX#Apple Silicon#local LLM#KV cache#inference engine
S
SGLang Project@huggingface·
👀86
🔥94

SGLang v0.5.22 Released: DeepSeek-V4.1 Throughput Jumps 2.3x on H200 and 1.9x on MI355X, InstantTensor Loads Models 10x Faster

The SGLang team released v0.5.22 featuring 950 PRs from 245 contributors: DeepSeek-V4.1 delivers up to 2.3x more throughput per GPU on NVIDIA H200 and 1.9x on AMD MI350X/MI355X via DSpark. The new InstantTensor loader cuts model loading time by 10x. B200 decodes Qwen3.8-Flash-Next 12% faster, while B300 speeds up GLM-5.3-Flash by 10% and yields 1.85x decode throughput for MiMo-V2.6-Pro. SGLang also adds native Apple Silicon PyTorch support and ComfyUI integration.

SGLang v0.5.22 Released: DeepSeek-V4.1 Throughput Jumps 2.3x on H200 and 1.9x on MI355X, InstantTensor Loads Models 10x Faster

Key Takeaways

  • SGLang v0.5.22 released with 950 PRs from 245 contributors across heterogeneous platforms
  • DeepSeek-V4.1 achieves up to 2.3x throughput per GPU on NVIDIA H200 and 1.9x on AMD MI350X/MI355X via DSpark
  • New InstantTensor loader speeds up large model checkpoint ingestion by 10x
  • Qwen3.8-Flash-Next decodes 12% faster on B200; MiMo-V2.6-Pro reaches 1.85x decode throughput on B300
  • PD disaggregation cuts p95 TTFT in half (-50%) on long prompts, alongside native Apple Silicon and ComfyUI diffusion support
Read full story
Linked Models & Agents:
#SGLang#DeepSeek-V4.1#AMD MI355X#B200#B300#Inference
S
Strands Agents (AWS)strands-agents·
👀91
🔥85

AWS open-sources Strands Box: OS isolation plus temporal Dogwood policies to rein in agents running in YOLO mode

On Oct 7 AWS released Strands Box v0.1.0, an open-source (Rust, Apache 2.0) sandbox for AI agents. It pairs OS-level isolation with the Dogwood policy language: shell, Python, outbound HTTP and MCP calls all pass through one default-deny policy engine with a shared event history, so rules can depend on earlier actions and elapsed time. A gateway injects credentials so the agent never sees the secrets. Preview supports macOS on Apple silicon only; Linux is in development.

AWS open-sources Strands Box: OS isolation plus temporal Dogwood policies to rein in agents running in YOLO mode

Key Takeaways

  • Release: v0.1.0 shipped 2026-10-07 17:24 UTC, written in Rust, Apache 2.0 (GitHub)
  • Default deny: engine-checked operations need a matching permit, and forbid overrides it (Dogwood)
  • Temporal rule example: an agent may post Slack status updates at most 3 times per 10 minutes; git push timing and API spend caps work the same way (The Register)
  • Platforms: macOS 15+ on Apple silicon only for now; Linux in development, Windows on the radar, AgentCore / ECS / Kubernetes planned
  • Benchmarks: AWS has not published performance or security benchmark numbers
Read full story
Linked Models & Agents:
🤖 Strands Agents🤖 Claude Code🌐 AWS
#AWS#Strands Box#Dogwood#Sandbox#Agent safety#MCP#Open source#Rust
G
Gökdeniz GülmezGoekdeniz-Guelmez·
👀83
🔥78

MLX-LM-LoRA v5.8.6: Apple Silicon fine-tuning library jumps from 3.1.3, adds DSLA preference alignment and critic-free KLPO RL

Gökdeniz Gülmez shipped MLX-LM-LoRA v5.8.6 on Oct 5 (GitHub Release and PyPI), the first release after 3.1.3. It adds DSLA with DPO/ORPO/CPO objectives and critic-free KLPO with token/sequence routes and several KL estimators, improves GRPO stability and microbatching for Online DPO, XPO, RLHF REINFORCE and PPO, and adds fast VJP for gated-delta layers. Minimums rise to mlx>=0.32.3, mlx_lm>=0.32.0 and Python>=3.11; license standardized on Apache 2.0.

Key Takeaways

  • Jump from v3.1.3 (Sep 16) straight to v5.8.6, merging nine PRs (#64, #94, #96–#102).
  • Two new modes: --train-mode dsla (--dsla-loss dpo|orpo|cpo, default --latent-weight 0.1) and --train-mode klpo (--klpo-route token|sequence, estimators mc/binary/topk/full).
  • KLPO skips GRPO group normalization, PPO ratio clipping and the reference model, reusing GRPO reward callbacks and generation/scoring.
  • Raised minimums: mlx>=0.32.3, mlx_lm>=0.32.0, Python >=3.11.
  • QAT now covers DSLA; new GitHub Pages site with a Python API reference.
Read full story
Linked Models & Agents:
🌐 apple
#MLX#MLX-LM-LoRA#Apple Silicon#Fine-tuning#DPO#RLHF#KLPO#DSLA
ADSponsored
U
Unsloth@unslothai·
👀99
🔥94

Unsloth v0.1.900-beta: local Laya Decision models + Skills editor; ~4.5× LTX-2.3 clips and up to 6.3× faster VAE decode

Unsloth shipped v0.1.900-beta on 2026-09-28 (PyPI unsloth 2026.9.12): Desktop runs Laya Decision models locally via a Jev-compatible /v1/systemone endpoint, adds a Skills CRUD editor and Library viewers for PDF/Office, and improves Apple Silicon with batched serving, structured outputs, and TurboQuant KV. Media pipelines claim ~4.5× LTX-2.3 clips, 1.7–6.3× VAE decode, and up to ~1 minute faster MiniMax-H3 first render with 25–29 GiB lower peak VRAM.

Unsloth v0.1.900-beta: local Laya Decision models + Skills editor; ~4.5× LTX-2.3 clips and up to 6.3× faster VAE decode

Key Takeaways

  • Shipped v0.1.900-beta — local Laya Decision API + Skills editor + Library viewers
  • LTX-2.3 ~4.5×; VAE decode 1.7–6.3×; MiniMax-H3 first render up to ~1 min faster, −25–29 GiB peak VRAM
  • Apple Silicon: batched serving, structured outputs, TurboQuant KV; ModelScope downloads
  • Docs: docs.unsloth.ai
Read full story
Linked Models & Agents:
⚡ Laya⚡ LTX-2.3⚡ MiniMax-H3⚡ Gemma-4🤖 Unsloth-Studio🌐 Unsloth
#Unsloth#Decision Model#Jev#Skills#Apple Silicon#MLX#LTX#Fine tuning
X
Xorbits AI@XorbitsAI·
👀66
🔥93

Xinference v3.5.0 Released: Native YuE2 Music Generation, MinerU2.5 PDF Parsing & Apple MLX Audio Models

Open-source distributed model serving engine Xinference (xorbitsai/inference) shipped v3.5.0, substantially upgrading its multimodal inference and edge acceleration capabilities. The release adds out-of-the-box support for YuE2 music generation, alongside AuK/AuK-Flash and Irodori TTS v4.1. On Apple Silicon, Xinference introduces native Apple MLX hardware acceleration for FishAudio-S2-Pro, Irodori TTS Small, and Breeze TTS 2. On the visual and document AI front, it integrates MinerU2.5 for complex document parsing and LingBot-World-V2 for embodied world simulation, while introducing distributed correlation metadata propagation across actor calls for enterprise observability.

Xinference v3.5.0 Released: Native YuE2 Music Generation, MinerU2.5 PDF Parsing & Apple MLX Audio Models

Key Takeaways

  • Generative audio expansion: native support for YuE2 music generation plus AuK, AuK-Flash, and Irodori TTS v4.1
  • Apple MLX acceleration: brings native MLX execution for FishAudio-S2-Pro, Irodori TTS Small, and Breeze TTS 2 on Apple Silicon
  • Document AI & World Models: out-of-the-box deployment for MinerU2.5 document parsing and LingBot-World-V2 world simulation
  • Distributed correlation tracing: propagates correlation metadata across actor calls for seamless multi-agent debugging
  • Enterprise audit controls: opt-in model request logging and refined virtualenv package management in the web dashboard
Read full story
Linked Models & Agents:
⚡ YuE2⚡ FishAudio-S2-Pro⚡ MinerU2.5⚡ LingBot-World-V2🤖 Xinference Worker🌐 Xorbits AI🌐 SenseTime
#Xinference#Model Serving#Apple MLX#YuE2#FishAudio#MinerU#Distributed Inference
O
Ollama@ollama·
👀114
🔥95

Ollama v0.40 Released: Defaults to Apple Native MLX Runner on Apple Silicon for Doubled Local Throughput

Local LLM inference runtime Ollama (@ollama) officially rolled out v0.40.0. The standout architectural milestone is the default adoption of Apple's native MLX framework on all Apple Silicon Mac devices (M1 through M5), replacing the legacy CPU/Metal compute paths. Supporting major architectures like Qwen 3.8 and Llama 3 natively, the update harnesses Apple's unified memory bandwidth, slashing time-to-first-token by 42% and nearly doubling continuous generation throughput without manual configuration.

Ollama v0.40 Released: Defaults to Apple Native MLX Runner on Apple Silicon for Doubled Local Throughput

Key Takeaways

  • Automatically routes model executions to Apple's native MLX framework by default on all Apple Silicon Mac hardware.
  • Fully unlocks unified memory bandwidth, boosting Qwen 3.8 and Llama 3 throughput by 85% while cutting TTFT by 42%.
  • Requires zero manual compiler flags or configuration switches; seamless upgrade via standard ollama run commands.
  • Cross-platform binary distributions and complete Git release changelog available directly on GitHub.
Read full story
Linked Models & Agents:
⚡ Qwen 3.8⚡ Llama 3.1 8b🤖 local-agent🌐 ollama🌐 apple
#Ollama#Apple Silicon#MLX#Local LLM#Performance#Release
I
Inco AI@inco_ai·
👀203
🔥93
View on 𝕏 ↗

Inco open-sources Splash: ~144 tok/s for Qwen3.8-27B on M5 Max

Inco AI open-sourced Splash, a local inference engine built for Apple silicon and models like Qwen3.8-27B. On an M5 Max MacBook Pro it reports about 144 tok/s—up to 3× Ollama, 2× oMLX, and nearly 4× when an agent fans out sub-agents. LM Studio shipped day-0 support under Settings > Runtime > Experimental backends.

Key Takeaways

  • Open-source Splash: local inference tuned for Apple silicon and Qwen3.8-27B.
  • Demo: ~144 tok/s on M5 Max; up to ~3× Ollama / 2× oMLX, nearly 4× with agent fan-out.
  • LM Studio day-0 support; repo github.com/incoai/splash; blog lmstudio.ai/blog/splash-engine.
Read full story
Linked Models & Agents:
⚡ Qwen3.8-27B🌐 inco🌐 Qwen (通义千问)🌐 lmstudio
#Inco#Splash#Qwen3.8-27B#Apple Silicon#M5 Max#LM Studio#local inference#open source
P
Perplexity@Perplexity_AI·
👀65
🔥91
View on 𝕏 ↗

Perplexity open-sources Lily — local inference engine for Qwen3.6 on Apple silicon

Perplexity (@Perplexity_AI) open-sources Lily, the local inference engine built for hybrid compute in Perplexity Computer. Lily is specialized for Qwen3.6-35B-A3B on Apple silicon so on-device compute does not bottleneck Computer tasks.

Key Takeaways

  • Official open-source of Lily for Perplexity Computer hybrid local/cloud compute.
  • Specialized local inference for Qwen3.6-35B-A3B on Apple silicon.
  • Goal: keep on-device inference from bottlenecking Computer agent tasks.
Read full story
Linked Models & Agents:
⚡ Qwen3.6-Plus🤖 Cursor Composer (Agent Mode)🤖 Claude Code (Anthropic)🌐 perplexity🌐 alibaba
#Lily#Perplexity#Qwen3.6#Apple silicon#local inference#open source
You've caught up with all stories