llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts
llama.cpp v0.6.0 is out, featuring the new llama_batch_ext API, major Metal backend speedups, web UI updates, and new ...
ggml v0.26.0 Released with Sparse Flash Attention & New Backends
ggml v0.26.0 is released, featuring sparse Flash Attention optimizations, expanded quantization support for WebGPU, a ...
vLLM v0.31.0 Released: Fast Restart and Hardware Optimization
vLLM v0.31.0 is released, featuring fast restart, DeepSeek optimizations, prefix cache security improvements, and bro ...
Local AI Model Weekly Highlights: Structured Output & Video LoRAs
Weekly roundup of local generative AI models: structured output models, video generation LoRAs, inference engines, an ...
DiarizationLM-Gemma-4-E4B-v1 Vision-Language Model: 8GB+ VRAM
Learn about DiarizationLM-Gemma-4-E4B-v1, an open-weights LLM for speech processing by Google based on Gemma 4 E4B, w ...
GLM-5.3-Flash-GGUF Vision-Language Model: ~150GB Memory
Discover ggml-org/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring v ...
Aleph Alpha Releases Kolibri: A New Open-Weight MoE Model
German AI firm Aleph Alpha released Kolibri, an open-weight Mixture-of-Experts model supporting English and German wi ...
LocalAI v4.11.0 Released with Failover and NeMo Audio Models
LocalAI v4.11.0 introduces model failover chains, NeMo speech-to-text models, decision models, signed OCI galleries, ...
Magnitude 0.2.4 Released: Faster Inference and Lower Memory
Magnitude 0.2.4 is released with significantly reduced memory usage, faster kernel tuning, and improved error handlin ...
SGLang v0.5.21 Released with Dynamic PD Role Switching
SGLang v0.5.21 is out, bringing dynamic PD role switching, a Rust-based prefix cache core, new APIs, and expanded mod ...