<\/p>\n
Values determined by this site’s code when the information was collected. Dates are JST.<\/em><\/p>\n<\/p>\n<\/p>\nWhat we checked ourselves<\/strong><\/p>\n\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/li>\n<\/ul>\nDetails and conditions are in \u201cOur Own Measurements\u201d below.<\/p>\n<\/p>\nOverview<\/h2>\nggml-org has released “ggml-org\/GLM-5.3-Flash-GGUF”, which is a GGUF quantized version of the open-weight multimodal model “GLM-5.3-Flash-BF16” published by zai-org.<\/p>\nThe original model, GLM-5.3-Flash, is the first native multimodal model in the GLM-5 series. It adopts a Mixture of Experts (MoE) configuration with a total parameter count of 320B and 18B active parameters. It is designed to balance efficient long-context processing and high performance by incorporating a hybrid architecture combining sparse and linear attention, along with Manifold-Constrained Hyper-Connections (mHC).<\/p>\nThis repository provides the model converted into the GGUF format for use in inference environments such as llama.cpp, including the mmproj for the vision encoder and the MTP sidecar for speculative sampling.<\/p>\nSpecifications<\/h2>\n\nTotal parameters: 320B<\/li>\nActive parameters: 18B<\/li>\nArchitecture: MoE (288 routed experts per layer, 1 shared expert), mHC + DSA, 1 MTP layer<\/li>\n<\/ul>\nPerformance<\/h2>\nSince this model is a converted version in GGUF format, performance evaluations are based on the public information of the original model “GLM-5.3-Flash-BF16”. Note that slight variations from the original precision may occur due to quantization.<\/p>\nAccording to explanations from the original model’s publishers, it delivers performance surpassing the previous-generation GLM-5.2 across various benchmarks and real-world workloads. Furthermore, it is reported to achieve performance levels approaching Claude Opus 4.8 on coding-related and agent-related benchmarks.<\/p>\nThe evaluation procedures for the original model cite the following benchmarks:<\/p>\n\nHLE w\/ tools (full set)<\/li>\nNL2Repo<\/li>\nDeepSWE<\/li>\nTerminal-Bench 2.1<\/li>\nAgent\u2019s Last Exam<\/li>\nToolathlon Verified<\/li>\nAutomationBench<\/li>\nGDPval-AA v2<\/li>\nBabyVision<\/li>\n<\/ul>\nThe original model features a parameter (reasoning_effort<\/code>) to control the amount of reasoning during inference, allowing users to adjust inference capabilities by specifying one of three stages: low<\/code>, high<\/code>, or max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
What we checked ourselves<\/strong><\/p>\n\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/li>\n<\/ul>\nDetails and conditions are in \u201cOur Own Measurements\u201d below.<\/p>\n<\/p>\nOverview<\/h2>\nggml-org has released “ggml-org\/GLM-5.3-Flash-GGUF”, which is a GGUF quantized version of the open-weight multimodal model “GLM-5.3-Flash-BF16” published by zai-org.<\/p>\nThe original model, GLM-5.3-Flash, is the first native multimodal model in the GLM-5 series. It adopts a Mixture of Experts (MoE) configuration with a total parameter count of 320B and 18B active parameters. It is designed to balance efficient long-context processing and high performance by incorporating a hybrid architecture combining sparse and linear attention, along with Manifold-Constrained Hyper-Connections (mHC).<\/p>\nThis repository provides the model converted into the GGUF format for use in inference environments such as llama.cpp, including the mmproj for the vision encoder and the MTP sidecar for speculative sampling.<\/p>\nSpecifications<\/h2>\n\nTotal parameters: 320B<\/li>\nActive parameters: 18B<\/li>\nArchitecture: MoE (288 routed experts per layer, 1 shared expert), mHC + DSA, 1 MTP layer<\/li>\n<\/ul>\nPerformance<\/h2>\nSince this model is a converted version in GGUF format, performance evaluations are based on the public information of the original model “GLM-5.3-Flash-BF16”. Note that slight variations from the original precision may occur due to quantization.<\/p>\nAccording to explanations from the original model’s publishers, it delivers performance surpassing the previous-generation GLM-5.2 across various benchmarks and real-world workloads. Furthermore, it is reported to achieve performance levels approaching Claude Opus 4.8 on coding-related and agent-related benchmarks.<\/p>\nThe evaluation procedures for the original model cite the following benchmarks:<\/p>\n\nHLE w\/ tools (full set)<\/li>\nNL2Repo<\/li>\nDeepSWE<\/li>\nTerminal-Bench 2.1<\/li>\nAgent\u2019s Last Exam<\/li>\nToolathlon Verified<\/li>\nAutomationBench<\/li>\nGDPval-AA v2<\/li>\nBabyVision<\/li>\n<\/ul>\nThe original model features a parameter (reasoning_effort<\/code>) to control the amount of reasoning during inference, allowing users to adjust inference capabilities by specifying one of three stages: low<\/code>, high<\/code>, or max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Details and conditions are in \u201cOur Own Measurements\u201d below.<\/p>\n
ggml-org has released “ggml-org\/GLM-5.3-Flash-GGUF”, which is a GGUF quantized version of the open-weight multimodal model “GLM-5.3-Flash-BF16” published by zai-org.<\/p>\n
The original model, GLM-5.3-Flash, is the first native multimodal model in the GLM-5 series. It adopts a Mixture of Experts (MoE) configuration with a total parameter count of 320B and 18B active parameters. It is designed to balance efficient long-context processing and high performance by incorporating a hybrid architecture combining sparse and linear attention, along with Manifold-Constrained Hyper-Connections (mHC).<\/p>\n
This repository provides the model converted into the GGUF format for use in inference environments such as llama.cpp, including the mmproj for the vision encoder and the MTP sidecar for speculative sampling.<\/p>\n
Since this model is a converted version in GGUF format, performance evaluations are based on the public information of the original model “GLM-5.3-Flash-BF16”. Note that slight variations from the original precision may occur due to quantization.<\/p>\n
According to explanations from the original model’s publishers, it delivers performance surpassing the previous-generation GLM-5.2 across various benchmarks and real-world workloads. Furthermore, it is reported to achieve performance levels approaching Claude Opus 4.8 on coding-related and agent-related benchmarks.<\/p>\n
The evaluation procedures for the original model cite the following benchmarks:<\/p>\n
The original model features a parameter (reasoning_effort<\/code>) to control the amount of reasoning during inference, allowing users to adjust inference capabilities by specifying one of three stages: low<\/code>, high<\/code>, or max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
reasoning_effort<\/code>) to control the amount of reasoning during inference, allowing users to adjust inference capabilities by specifying one of three stages: low<\/code>, high<\/code>, or max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
low<\/code>, high<\/code>, or max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
high<\/code>, or max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
max<\/code> (default).<\/p>\nStrengths and Use Cases<\/h2>\nThe base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The base model “GLM-5.3-Flash-BF16” is the first in the GLM-5 series to support native multimodal processing. It supports image-text input and conversation processing (image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
image-text-to-text<\/code>), making it useful for general tasks involving visual information.<\/p>\nThe main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\nCoding and Autonomous Agent Support<\/h3>\nThe original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\nEfficient Long-Context Processing<\/h3>\nArchitecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\nThinking Budget Control and Dialogue<\/h3>\nThe original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The main strengths and expected use cases derived from the original model’s design and evaluation details are as follows:<\/p>\n
The original model is optimized to demonstrate high capabilities in coding and agent-related benchmarks. Evaluations adopt “NL2Repo” for repository generation, “DeepSWE” for autonomously solving software engineering tasks, “Terminal-Bench 2.1” for evaluating terminal operations, “Toolathlon Verified” for measuring tool utilization capabilities, and “AutomationBench” for verifying automated workflows. This makes it suitable for complex agent use cases such as not only creating and editing code, but also executing development tools and automating autonomous workflows.<\/p>\n
Architecturally, a hybrid structure combining sparse and linear attention is introduced. This aims to significantly reduce inference overhead when handling long contexts while maintaining context-grasping capabilities. It is suited for scenarios where context lengths tend to grow large, such as analyzing massive codebases or handling agent tasks that include multi-step tool-calling histories.<\/p>\n
The original model features a parameter (reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
reasoning_effort<\/code>) that controls the amount of reasoning during inference, allowing users to adjust inference cost and depth according to their use case. Additionally, the chat template provides a clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
clear_thinking<\/code> option to control the display of the thought process, and explicitly setting clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
clear_thinking=true<\/code> is recommended for general conversational use. Supported language tags include English (en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
en<\/code>) and Chinese (zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
zh<\/code>).<\/p>\nHigh-Speed Inference in Local Environments<\/h3>\nIn this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
In this GGUF version, in addition to the mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
mmproj<\/code> file for the vision encoder, an MTP (Multi-Token Prediction) sidecar that enables speculative decoding is included. This is expected to achieve faster text generation than standard inference when using a compatible inference engine.<\/p>\n<\/p>\nHardware Requirements<\/h2>\nEstimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 321.3B parameters (taken from the base model zai-org\/GLM-5.3-Flash-BF16)<\/p>\n\n\n\n\nYour VRAM<\/th>\nQuantization<\/th>\nFile size<\/th>\nEst. memory needed<\/th>\n<\/tr>\n<\/thead>\n\n\nMore than 150GB of VRAM (multi-GPU or CPU offload required)<\/td>\nQ2_K<\/td>\n124.6GB<\/td>\n149.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nMemory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference<\/a>. What the quantization names mean: glossary<\/a>.<\/em><\/p>\n<\/p>\n<\/p>\nCan You Run It Locally?<\/h2>\nRuns in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Runs in Ollama, LM Studio and llama.cpp as-is.<\/strong><\/p>\nIt is distributed in GGUF, so no conversion is needed.<\/p>\nLicense \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
It is distributed in GGUF, so no conversion is needed.<\/p>\n
License \u2014 other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
other<\/code>:<\/strong> A custom license from the publisher. Check the original terms directly, including whether commercial use is permitted.<\/p>\nCompression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Compression:<\/strong> the Q2_K build measures 3.33 bits per weight \u2014 about 21% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\nCompiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found” means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher’s original terms before relying on them.<\/em><\/p>\n<\/p>\n<\/p>\nOur Own Measurements<\/h2>\nValues we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Values we checked ourselves on our server (no GPU) by actually reading this model’s files, without running the model \u2014 not figures copied from the model card. How we measure, and the results for every model: Our Measurements<\/a>.<\/p>\nMeasurements We Did Not Take<\/h3>\nWe have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
We have not confirmed that this site meets the commercial-use terms of this model’s license (other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
other<\/code>). Because this site carries advertising, we did not run the model (no CPU run, answers, quantization comparison, conversion or generation). Only values we checked without running the model, such as token counts and the GGUF header, are shown.<\/p>\nJapanese Token Efficiency<\/h3>\n\n\n\n\nTokenizer<\/th>\nTokens per 1,000 Japanese characters<\/th>\nRatio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n\n\nThis model<\/strong><\/td>\n733<\/strong><\/td>\n1.34\u00d7<\/strong><\/td>\n<\/tr>\n\nQwen3<\/td>\n688<\/td>\n1.26\u00d7<\/td>\n<\/tr>\n\nLlama 3.2<\/td>\n744<\/td>\n1.36\u00d7<\/td>\n<\/tr>\n\nGemma 3<\/td>\n564<\/td>\n1.03\u00d7<\/td>\n<\/tr>\n\ngpt-oss<\/td>\n795<\/td>\n1.45\u00d7<\/td>\n<\/tr>\n\nLLM-jp-3<\/td>\n497<\/td>\n0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nIt needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\nCounted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
It needs about 7% more tokens than the Qwen3 tokenizer for the same Japanese text, so only about 0.94\u00d7 as much Japanese fits in the same context length.<\/p>\n
Counted with the tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
tokenizer.json<\/code> of zai-org\/GLM-5.3-Flash-BF16<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\nInside the GGUF File<\/h3>\nFile: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
File: GLM-5.3-Flash-Q2_K-00001-of-00002.gguf<\/a> (124.62GB, Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Q2_K<\/code>). We read only the header (metadata) of the file, not the weights.<\/p>\n\n\n\n\nItem<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n\n\nArchitecture (as named in the GGUF)<\/td>\nglm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
glm5-next<\/code><\/td>\n<\/tr>\n\nMaximum trained context length<\/td>\n1,048,576 tokens<\/td>\n<\/tr>\n\nLayers<\/td>\n45<\/td>\n<\/tr>\n\nExperts<\/td>\n8 active out of 288<\/td>\n<\/tr>\n\nVocabulary size<\/td>\n154,880<\/td>\n<\/tr>\n\nChat template<\/td>\nIncluded (mentions tool calls, has a thinking switch)<\/td>\n<\/tr>\n\nimatrix<\/td>\nNot recorded in the file<\/td>\n<\/tr>\n\nWeight types (share of parameters)<\/td>\nQ2_K 64.8% \/ Q4_K 32.4% \/ Q8_0 2.7% \/ BF16 0.1% \/ other 0.0%<\/td>\n<\/tr>\n\nAverage bits per weight<\/td>\n3.42 bits<\/td>\n<\/tr>\n\nEmbedding \/ output layer type<\/td>\nQ8_0 \/ Q8_0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\nThe quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n<\/p>\n<\/p>\nRecent Models in the Same Size Class<\/h2>\nModels with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The quant name in the file name describes the file as a whole; in practice layers mix several types. The average bits per weight is the measured file data divided by the number of weights.<\/p>\n
Models with <\/em>over 40B<\/em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.<\/em><\/p>\n\n\u2192 Scroll horizontally to see all columns<\/p>\n\n\n\nModel<\/th>\nParameters<\/th>\nSmallest VRAM tier<\/th>\nLicense<\/th>\nOur article<\/th>\n<\/tr>\n<\/thead>\n\n\nQwen\/Qwen3.8-Flash-Next<\/td>\n180.0B<\/td>\n\u2014<\/td>\nqwen-community-1.0<\/td>\nQwen3.8-Flash-Next Multimodal MoE Model: ~402GB Memory<\/a> (2026-10-04)<\/td>\n<\/tr>\n\nbartowski\/Intern-S2-397B-GGUF<\/td>\n403.4B<\/td>\n\u2014<\/td>\napache-2.0<\/td>\nIntern-S2-397B-GGUF Vision-Language Model: ~102GB Memory<\/a> (2026-09-15)<\/td>\n<\/tr>\n\ndeepseek-ai\/DeepSeek-V4.1-Flash<\/td>\n763.2B<\/td>\n\u2014<\/td>\nmit<\/td>\nDeepSeek-V4.1-Flash 552B Multimodal MoE Model: ~570GB Memory<\/a> (2026-09-10)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/p>\nHow to Get It<\/h2>\nThis model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\nWhen using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\u2192 Scroll horizontally to see all columns<\/p>\n
This model is distributed in GGUF format on the Hugging Face repository “ggml-org\/GLM-5.3-Flash-GGUF”. Since it is not a gated model, you can download and use it without prior application or consent procedures.<\/p>\n
When using CLI tools from the official llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
llama.cpp<\/code> family, you can start a server by directly specifying the Hugging Face repository with the following command:<\/p>\nllama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
llama serve -hf ggml-org\/GLM-5.3-Flash-GGUF\n<\/code><\/pre>\nThe internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n\nQ4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
The internal specifications of the quantized versions provided in this repository have the following features:<\/p>\n
Q4_K<\/code>: All routed experts are quantized with Q4_K.<\/li>\nQ2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Q2_K<\/code>: Composed of Q4_K for routed down experts and Q2_K for gate\/up experts.<\/li>\nSmaller non-expert tensors, such as embedding layers, attention layers, shared experts, and dense FFNs, are kept as Q8_0 to maintain quality.<\/li>\nBundled with mmproj (Q8_0) for the vision encoder and MTP sidecars (Q8_0 \/ Q4_0) for speculative sampling.<\/li>\nNote that low-bit quantization is noted as not having undergone calibration using imatrix (importance matrix).<\/li>\n<\/ul>\nRegarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
Regarding the license classification, the base model “GLM-5.3-Flash-BF16” is set under the MIT license, but this GGUF repository carries an other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
other<\/code> tag. Please check the repository’s license terms in advance before use.<\/p>\n<\/p>\nRelated Articles<\/h2>\n\nOpenJev-GGUF 27.4B Decision-Making Specialized Model: 24GB+ VRAM<\/a><\/li>\nMiMo-V2.6-Flash-RL-GGUF Multimodal MoE Model: ~141GB Memory<\/a><\/li>\nTesting OpenJev GGUF: A Tiny Model for Runtime Loaders<\/a><\/li>\nMiMo-V2.6-Distill-Qwen-9B-GGUF: Our Test Answers, 12GB+ VRAM<\/a><\/li>\n<\/ul>\n<\/p>\n<\/p>\nWhat to Read Next<\/h2>\n\nFind models by VRAM<\/strong> (This model does not fit even in 80GB; its smallest build needs about 150GB) \u2192 VRAM quick reference, including models over 80GB<\/a><\/li>\nEngines that run this model<\/strong> \u2192 llama.cpp<\/a> \/ Ollama<\/a><\/li>\nWhat Q2_K mean and where to get this model<\/strong> \u2192 GGUF format guide and models<\/a> \/ Quantization and model-format glossary<\/a><\/li>\nLearn about the publisher<\/strong> \u2192 Z.ai (Zhipu AI): models, licenses and articles<\/a><\/li>\nOther models for the same task<\/strong> \u2192 Other vision-language models<\/a><\/li>\n<\/ul>\n<\/p>\nSources<\/h2>\n\nggml-org\/GLM-5.3-Flash-GGUF: https:\/\/huggingface.co\/ggml-org\/GLM-5.3-Flash-GGUF<\/a><\/li>\nzai-org\/GLM-5.3-Flash-BF16: https:\/\/huggingface.co\/zai-org\/GLM-5.3-Flash-BF16<\/a><\/li>\n<\/ul>\n<\/p>\nUpdate History<\/h2>\n\n2026-10-05: Added our own measurements: Japanese token efficiency, what is inside the GGUF file.<\/li>\n<\/ul>\n<\/p>\n","protected":false},"excerpt":{"rendered":"Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
<\/p>\n","protected":false},"excerpt":{"rendered":"
Discover ggml-org\/GLM-5.3-Flash-GGUF, a GGUF quantization of zai-org’s native multimodal MoE model, featuring vision encoders and MTP sidecars.<\/p>\n","protected":false},"author":1,"featured_media":9654,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[161,1970,163,1469,2950,1073,165,1547,1952],"class_list":["post-9655","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-ggml-en","tag-ggml-org-en","tag-gguf-en","tag-glm-5-3-flash-en","tag-glm-5-3-flash-gguf-en","tag-llama-cpp-en","tag-moe-en","tag-verified","tag-vlm-en"],"lang":"en","translations":{"en":9655,"ja":9653},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=9655"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions"}],"predecessor-version":[{"id":9905,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/9655\/revisions\/9905"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/9654"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=9655"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=9655"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=9655"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}