Mini-AGI: A Byte-Level Continual Learning Model for 8GB VRAM
Discover mini-AGI, an experimental byte-level language model designed to run on an 8GB VRAM GPU with continuous learn ...
GLM Announces Custom Inference Infrastructure and Local MoE Tests
GLM announces a new custom inference infrastructure built on Chinese accelerators, alongside community tests running ...
llama.cpp v0.4.1 Released with Breaking Changes
llama.cpp v0.4.1 is released, featuring important argument cleanups, new model architectures, and stability improveme ...
ExLlamaV3 1.5.0 Released: Major MoE Inference Speedups
ExLlamaV3 1.5.0 is released, bringing massive MoE inference speedups, experimental Linux CPU offloading, and Nemotron ...
Edge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM
Edge0 released Edge0-35B-A3B-preview, a 35B sparse MoE model running on phone-class memory using streaming inference. ...
DeepSeek-V4-Pro-0813-nvfp4-DSpark: ~1005GB Memory
NVIDIA has released nvidia/DeepSeek-V4-Pro-0813-nvfp4-DSpark, a fully NVFP4-quantized MoE language model supporting S ...
DeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory
deepseek-ai has released DeepSeek-V4.1-Flash on Hugging Face, a multimodal MoE model supporting up to 1M tokens. Lear ...
vLLM v0.29.0 Released: Model Runner V2 Default and More
vLLM v0.29.0 is released with Model Runner V2 as default, FlashInfer all-reduce, new admission control flags, and sup ...
Nex-N2.5-mini Text Generation Model: Our Test Answers, 16GB+ VRAM
Discover the specs, performance, and hardware requirements for Nex-N2.5-mini, an open-weight multimodal model by Nex- ...
Inference Engine Updates and Practical GGUF Models Like Qwopus
Explore the latest local AI trends, including inference engine updates like ggml v0.23.0 and ExLlamaV3, and GGUF rele ...