vLLM v0.31.0 Released: Fast Restart and Hardware Optimization
vLLM v0.31.0 is released, featuring fast restart, DeepSeek optimizations, prefix cache security improvements, and bro ...
Qwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory
Explore Qwen3.8-Flash-Next, a next-gen open-weight multimodal causal language model featuring hybrid attention, MoE a ...
Testing OpenJev GGUF: A Tiny Model for Runtime Loaders
Explore tinyopenjev-for-testing-gguf, a miniature GGUF model by ggml-org designed for testing loader and runtime beha ...
OrcaSAQ-2-27B Text Generation Model: Our Test Answers, 16GB+ VRAM
OrcaRouter has released OrcaSAQ2 27B, a compressed 3.21-bit quantization of Qwen3.8-27B designed to run on a single 1 ...
Hemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds
Altworld has released Hemmingway-1, a 27B language model fine-tuned for daily human-like writing, available on Huggin ...
DeepSeek-V4-Pro-0813 Text Generation Model: ~998GB Memory, GGUF Builds
Discover DeepSeek-V4-Pro-0813, a 1.6T MoE model on Hugging Face featuring enhanced agent capabilities, 1M context, an ...
K2-Horizon-32B-NVFP4: Our Test Answers, 32GB+ VRAM
IFM releases K2-Horizon-32B-NVFP4, an NVFP4 quantized version of the 32B long-context open-weight LLM optimized for N ...
K2-Horizon-375B-A23B-NVFP4 Text Generation Model: ~257GB Memory
Explore the NVFP4 quantized version of IFM’s K2-Horizon-375B-A23B MoE model, designed for NVIDIA Blackwell GPUs ...
vLLM v0.30.0 Released: Fast Start Weight Caching and New Models
vLLM v0.30.0 is released, featuring Fast Start weight caching, engine initialization speedups, DeepSeek-V4.1-Flash su ...
LocalAI v4.10.0 Released: Fleet Dashboard & M5 Support
LocalAI v4.10.0 is out, featuring a fleet management dashboard, Apple M5 startup crash fixes, credential file managem ...