{"id":11912,"date":"2026-11-17T11:00:48","date_gmt":"2026-11-17T02:00:48","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/?p=11912"},"modified":"2026-11-17T11:09:54","modified_gmt":"2026-11-17T02:09:54","slug":"meta-muse-spark-open-2","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/meta-muse-spark-open-2\/","title":{"rendered":"Meta Releases Muse Spark Open: Multimodal MoE Model"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Publisher<\/td>\n<td>Hugging Face Blog<\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-10-15<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Unverified (not confirmed by a primary source)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>It is reported that Meta has released &#8220;Muse Spark Open&#8221;, an open-weight state-of-the-art multimodal Mixture-of-Experts (MoE) model. However, this information has not been officially confirmed at this time.<\/p>\n<p>This model can handle text, images, audio, and video within a single model, providing developers with a common interface to ask questions across various types of inputs. Muse Spark Open is the full-size model in the &#8220;Muse Glimmer&#8221; series of smaller dense models previously released by Meta. Both models use the same tokenizer and vocabulary, with Muse Spark Open introducing the MoE architecture at a larger scale.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Total parameters: Approximately 816 billion<\/li>\n<li>Active parameters per token: Approximately 42 billion<\/li>\n<li>Architecture: Mixture-of-Experts (MoE)<\/li>\n<li>Number of layers: 62 layers<\/li>\n<li>Hidden layer dimension: 8192<\/li>\n<li>Number of query heads: 128<\/li>\n<li>Number of key-value heads: 16 (Grouped-query attention)<\/li>\n<li>Head dimension: 64<\/li>\n<li>MoE configuration: 256 experts per layer, top-8 selected per token<\/li>\n<li>Context length: Up to 1 million tokens<\/li>\n<li>Input formats: Text, images, audio, video<\/li>\n<li>Output format: Text<\/li>\n<li>Input and output embedding layers: Independent matrices of approximately 1.66 billion parameters each<\/li>\n<li>Normalization methods: QK-norm and a normalization layer on the output layer<\/li>\n<li>Residual connections: Combination using learned residual gates<\/li>\n<li>Perception Encoder parameters: 1.8 billion<\/li>\n<li>Perception Encoder layers: 50 layers<\/li>\n<li>Perception Encoder hidden dimension: 1536<\/li>\n<li>Perception Encoder attention heads: 16<\/li>\n<li>Audio Encoder structure: Convolutional pyramid<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>According to the model card description, Muse Spark Open is a model optimized for coding. It has been trained on real development workflows such as multi-file refactoring and long debugging sessions. Specifically, it is designed to read entire codebases, write and debug code, and even execute tool calls.<\/p>\n<p>It is also specialized for agentic capabilities, capable of handling long-term tasks involving planning, execution, observation, and self-correction. This includes web browsing using images.<\/p>\n<p>Due to its support for multimodal inputs, it is reported to allow developers to work while understanding diverse contexts, such as reading screenshots, listening to voice memos, or watching screen recordings while working on code. This reportedly enables developers to give the model advanced instructions combining not only text but also visual and auditory information.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>Muse Spark Open is reported to feature capabilities as a coding-focused agent. Specifically, it is designed to handle tasks based on real-world development workflows, such as reading codebases, writing code, debugging, and tool calling. It is said to be capable of handling complex tasks like multi-file refactoring and extended debugging sessions.<\/p>\n<p>It also features powerful multimodal capabilities, allowing it to handle text, images, audio, and video simultaneously. This makes it possible to follow advanced instructions combining visual and auditory information while checking screenshots, listening to voice memos, or watching screen recordings during coding work. Furthermore, the execution of long-term tasks involving planning, execution, observation, and self-correction, including web browsing using images, is anticipated.<\/p>\n<h2>How to Get It<\/h2>\n<p>The model is reported to be distributed in the following formats:<\/p>\n<ul>\n<li>Distribution formats: BF16, NVFP4 (for NVIDIA Blackwell GPUs)<\/li>\n<li>Installation commands:<\/li>\n<li>When using llama.cpp: <code>bash curl -LsSf https:\/\/llama.app\/install.sh | sh llama serve -hf ggml-org\/Muse-Spark-Open-816B-A42B-GGUF:Q2_K<\/code> * When using SGLang: <code>bash export MODEL_ID=meta-models\/Muse-Spark-Open-816B-A42B-NVFP4 export CKPT=\"$(hf download \"$MODEL_ID\")\" SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=1 \\ uv run sglang_meta serve \\ --model-path \"$CKPT\" \\ --served-model-name \"$MODEL_ID\" \\ --quantization modelopt_fp4 \\ --moe-runner-backend flashinfer_trtllm \\ --tp-size 4 \\ --ep-size 4 \\ --chat-template \"$CKPT\/muse_spark_1_2_oss.jinja\" \\ --port 30000<\/code><\/li>\n<li>Supported engines: llama.cpp, SGLang, vLLM (NVFP4 requires the Blackwell series)<\/li>\n<\/ul>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/mistral-large-4-released-2\/\">Mistral Large 4 Released: Multimodal MoE Model<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/25\/liquidai-lfm2-5-vl-dspark-released\/\">LFM2.5-VL-3B-DSpark Draft Model for Vision-Language Models: 4GB+ VRAM<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/botpsy-0-8b-agentic-model\/\">BotPsy: 0.8B On-Device Agentic Model by Tether AI Research<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/11\/17\/anylanguagemodel-1-0-released\/\">AnyLanguageModel 1.0 Released for Apple Platforms<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/blog\/meta-muse-spark-open\"><a href=\"https:\/\/huggingface.co\/blog\/meta-muse-spark-open\">https:\/\/huggingface.co\/blog\/meta-muse-spark-open<\/a><\/a><\/li>\n<\/ul>\n<blockquote>\n<p><strong>This article contains unverified information.<\/strong> We will append an update note once it is confirmed by a primary source.<\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>Meta has reportedly released Muse Spark Open, an 816B parameter multimodal Mixture-of-Experts model supporting text, image, audio, and video.<\/p>\n","protected":false},"author":1,"featured_media":11911,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[163,1907,3166,165,3487,169,1565,592],"class_list":["post-11912","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-gguf-en","tag-hugging-face-blog-en","tag-meta-en","tag-moe-en","tag-muse-spark-open-en","tag-sglang-en","tag-unverified","tag-vllm-en"],"lang":"en","translations":{"en":11912,"ja":11910},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11912","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=11912"}],"version-history":[{"count":2,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11912\/revisions"}],"predecessor-version":[{"id":17809,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/11912\/revisions\/17809"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/11911"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=11912"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=11912"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=11912"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}