llama.cpp v0.6.0 Released with llama_batch_ext and Metal Boosts
llama.cpp v0.6.0 is out, featuring the new llama_batch_ext API, major Metal backend speedups, web UI updates, and new ...
ggml v0.25.2 Released with Hardware Backend Optimizations
ggml v0.25.2 has been released, focusing on improvements to hardware backends including CUDA, Vulkan, OpenCL, and Hex ...
llama.cpp v0.5.0 Released with Backend and Server Upgrades
llama.cpp v0.5.0 is released, featuring CUDA/Metal hardware acceleration, multi-address server support, and new model ...
ggml v0.24.0 Released with Backend Improvements and API Updates
ggml v0.24.0 is out, featuring a new precision control API, major backend improvements for Vulkan, SYCL, and Metal, a ...
ExLlamaV3 1.4.8 Released: VRAM and FA2 Updates
ExLlamaV3 1.4.8 is released. Key updates include VRAM allocation optimization and the complete removal of Flash Atten ...