Inference Engine Updates and Practical GGUF Models Like Qwopus

September 18, 2026

Inference Engine Updates and Practical GGUF Models Like Qwopus

Highlights of the Week

This week focused on inference engine updates that significantly enhance inference efficiency and distributed processing in local environments, alongside the release of GGUF format models that are easy to run locally.

  1. Major Feature Additions and Optimizations in Key Inference Engines
    Infrastructure development to push local compute resources to their limits progressed, including sparse attention and asynchronous RPC support in ggml v0.23.0, and successive updates to ExLlamaV3 (v1.4.7 and v1.4.8) improving MoE processing performance and VRAM optimization.

  2. GGUF Deployment of Practical Models
    GGUF versions of Qwopus3.8-27B-Flash, marketed for agents, and K2-Horizon-MoVA-36B-A4B, which adopts a MoE architecture, were released, broadening the options available for lightweight environments like llama.cpp.

  3. Progress in Expansion and Decentralization of Local Computation
    The NVIDIA PAIR virtual inference router was released, presenting a mechanism for computational expansion in local environments and distributed processing for AI agents.

Trends by Category

Engines and Tools

It was a week where improvements in inference efficiency and adaptation to new model structures progressed simultaneously.

SGLang released v0.5.19, adding support for Qwen3.8 and new diffusion models. ggml v0.23.0 introduced support for sparse attention and asynchronous RPC, advancing adaptability to distributed environments and long-context processing. Additionally, the high-speed inference library ExLlamaV3 received two updates in a short period. v1.4.7 improved CPU MoE performance and added the DRY sampler, while the subsequent v1.4.8 implemented VRAM optimization and the complete removal of the FlashAttention-2 code path. Furthermore, the release of the NVIDIA PAIR virtual inference router to expand local computation increased interest in the overall optimization of inference frameworks and distributed processing.

In image generation tools, InvokeAI released v6.14.1, adding memory management improvements and Krea-2 LoRA support. VRAM consumption optimization has progressed, aiming to stabilize image generation workflows in local environments.

Text Generation

Releases of models in quantization formats geared toward practical deployment stood out.

As GGUF formats that are easy to handle in the llama.cpp environment, the lightweight agent-focused model “Qwopus3.8-27B-Flash" and the IFM MoE model “K2-Horizon-MoVA-36B-A4B" have started becoming available. Beyond merely releasing base models, the availability of GGUF versions ready to run immediately on local PCs and servers is establishing a local experimentation environment.

Image, Video, and Audio

In the media generation field, the video generation model “WarmBloodAban/Minimax-h3_Singularity" was released on Hugging Face, drawing interest in Text-to-Video utilization via ComfyUI and other tools.

Community

Exploration of practical integration methods utilizing local models was also observed. A method linking coding agents with the 3DCG software Blender in a macOS environment became a hot topic in the community. However, this is a community-reported item containing unverified information that has not been officially confirmed. It is drawing attention as an example demonstrating interest in automating local tasks.

Articles This Week

Sources

Weekly Roundupexllamav3,ggml,GGUF,MoE,Qwopus,SGLang,確認済み