SGLang v0.5.20 Released: New Models and Optimizations
SGLang v0.5.20 is released with 713 PRs, adding support for new models, ROCm load time improvements, Intel XPU offici ...
needle3 Text Generation Model: 4GB+ VRAM
Cactus Compute has released Needle 3, a tiny 2bit automation foundation model for mobile, edge, and IoT devices focus ...
Unsloth v0.1.811-beta Released with AMD & NVIDIA Docker Images
Unsloth v0.1.811-beta is out, introducing new NVIDIA and AMD Docker images, multi-user accounts, 2x faster diffusion ...
Ternary-Bonsai-2-27B-gguf: Our Test Answers, 8GB+ VRAM
Discover Ternary-Bonsai-2-27B-gguf, a ternary language model by Prism ML achieving 1.72 bits/weight with near FP16 pe ...
Unsloth Now Supports Windows ARM64 for Local LLMs
Unsloth releases Windows ARM64 binaries, enabling local LLM execution and fine-tuning on ARM-based Windows devices.
Challenges of Migrating AI Agents to Local Ollama Environments
Discussing the technical hurdles, context thrashing, and prompt design issues when moving large agents from commercia ...
llama.cpp v0.4.1 Released with Breaking Changes
llama.cpp v0.4.1 is released, featuring important argument cleanups, new model architectures, and stability improveme ...
ExLlamaV3 1.5.0 Released: Major MoE Inference Speedups
ExLlamaV3 1.5.0 is released, bringing massive MoE inference speedups, experimental Linux CPU offloading, and Nemotron ...
litelm: Lightweight LLM Routing Library Released
Discover litelm, a lightweight LLM routing library extracting core routing and message format translation features fr ...
DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory
NVIDIA has released nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark, a fully NVFP4-quantized MoE language model supporting S ...