magnitudedev/magnitude v0.2.6 Released: Major Mac Speedups
magnitudedev/magnitude v0.2.6 is out, bringing massive prompt processing speedups and stability fixes for Apple Silic ...
llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts
llama.cpp v0.6.0 is out, featuring the new llama_batch_ext API, major Metal backend speedups, web UI updates, and new ...
ggml v0.26.0 Released with Sparse Flash Attention & New Backends
ggml v0.26.0 is released, featuring sparse Flash Attention optimizations, expanded quantization support for WebGPU, a ...
Magnitude CLI v0.2.5 Released: M5 Mac & MoE Speedups
Version 0.2.5 of @magnitudedev/cli is out, bringing 2x faster prompt processing on M5 Macs, MoE speedups, and M1/M2 f ...
Magnitude 0.2.4 Released: Faster Inference and Lower Memory
Magnitude 0.2.4 is released with significantly reduced memory usage, faster kernel tuning, and improved error handlin ...
llama.cpp v0.5.0 Released with Backend and Server Upgrades
llama.cpp v0.5.0 is released, featuring CUDA/Metal hardware acceleration, multi-address server support, and new model ...
Run GGUF Models Directly in Transformers with llama.cpp Support
Hugging Face adds direct execution support for llama.cpp GGUF models in transformers, starting with local Qwen3.5 inf ...
Open-Weight Model Weekly: Ternary 27B Models and Apple Silicon Updates
Weekly roundup of open-weight generative models: ternary quantization for 27B models, Apple Silicon optimizations, an ...
Inco AI Releases Splash Engine for Fast Local LLMs on Mac
Inco AI releases Splash Engine, an open-source local LLM inference engine optimized for Apple Silicon, natively integ ...
Qwen-2.5-1B-RLCD Text Generation Model: 4GB+ VRAM
Explore harshatheg/Qwen-2.5-1B-RLCD, an MLX implementation using parallel constrained decoding on Apple Silicon for h ...