{"id":543,"date":"2026-09-11T12:14:06","date_gmt":"2026-09-11T03:14:06","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/11\/edge0-35b-a3b-preview-sparse-moe\/"},"modified":"2026-11-03T16:08:25","modified_gmt":"2026-11-03T07:08:25","slug":"edge0-35b-a3b-preview-sparse-moe","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/11\/edge0-35b-a3b-preview-sparse-moe\/","title":{"rendered":"Edge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview\">Edge0\/Edge0-35B-A3B-preview<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-08<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-mlx-en\/\">MLX<\/a> \/ safetensors<\/td>\n<\/tr>\n<tr>\n<td>Paper<\/td>\n<td><a href=\"https:\/\/arxiv.org\/abs\/2609.18063\">arXiv:2609.18063<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<p><!-- lmw:lab-summary --><\/p>\n<p><strong>What we checked ourselves<\/strong><\/p>\n<ul>\n<li>It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/li>\n<\/ul>\n<p>Details and conditions are in \u201cOur Own Measurements\u201d below.<\/p>\n<p><!-- \/lmw:lab-summary --><\/p>\n<h2>Overview<\/h2>\n<p>Edge0 has released &#8220;Edge0-35B-A3B-preview&#8221;, a 35B-class sparse MoE model that runs in smartphone-class memory environments. It is a preview version that operates by integrating a 4-bit quantized model with LoRA and prerouter adapters using the streaming inference framework &#8220;edge0&#8221; designed for distribution.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Architecture: Qwen3_5MoeForConditionalGeneration (Qwen3_5_MoE)<\/li>\n<li>Number of parameters: 34.7B<\/li>\n<li>Number of layers: 40<\/li>\n<li>Number of experts \/ Active per token: 256 \/ 4 (K=4)<\/li>\n<li>Hidden size: 2048<\/li>\n<li>License: Apache 2.0<\/li>\n<li>Framework: edge0 (MLX backend)<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>According to measurements by the publishers, the average drop is kept to 3.9 points compared to the fp16 base model Qwen3.5-MoE 35B-A3B. The scores for various benchmarks are as follows.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Benchmark<\/th>\n<th style=\"text-align: right;\">edge0-35b (int4)<\/th>\n<th style=\"text-align: right;\">Qwen3.5-MoE 35B-A3B (fp16)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>AIME 2026<\/td>\n<td style=\"text-align: right;\">86.6<\/td>\n<td style=\"text-align: right;\">92.7<\/td>\n<\/tr>\n<tr>\n<td>HumanEval<\/td>\n<td style=\"text-align: right;\">90.9<\/td>\n<td style=\"text-align: right;\">95.1<\/td>\n<\/tr>\n<tr>\n<td>GPQA-Diamond<\/td>\n<td style=\"text-align: right;\">79.8<\/td>\n<td style=\"text-align: right;\">81.8<\/td>\n<\/tr>\n<tr>\n<td>MMLU-Pro<\/td>\n<td style=\"text-align: right;\">81.0<\/td>\n<td style=\"text-align: right;\">84.6<\/td>\n<\/tr>\n<tr>\n<td>IFBench<\/td>\n<td style=\"text-align: right;\">57.9<\/td>\n<td style=\"text-align: right;\">61.7<\/td>\n<\/tr>\n<tr>\n<td><strong>Average<\/strong><\/td>\n<td style=\"text-align: right;\"><strong>79.2<\/strong><\/td>\n<td style=\"text-align: right;\"><strong>83.2<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>The measured performance is as follows.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Decode speed<\/th>\n<th>Prefill throughput (cold \/ warm)<\/th>\n<th>Peak active memory*<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>14.9\u201317.7 tok\/s<\/td>\n<td>113 \/ 140 tok\/s<\/td>\n<td>2.9 GiB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>*Measured values for short contexts. KV cache is added for long texts. Expert weights are streamed on-demand from the SSD and are not resident.<\/p>\n<p>From these scores, it can be seen that the quantization loss due to int4 conversion is kept small, maintaining performance levels close to the base model in fields such as competitive mathematics (AIME), coding (HumanEval), difficult science questions (GPQA Diamond), and general knowledge\/reasoning (MMLU-Pro). On the other hand, as a limitation of the preview version, optimization for agent-based tasks is not yet advanced.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<ul>\n<li>Edge and on-device inference where GPU VRAM is depleted but fast storage (NVMe or internal flash) is available<\/li>\n<li>Batch serving on a single host machine (sharing multiple LoRA adapter sets with a single read-only base without re-quantization)<\/li>\n<li>Multilingual chat and reasoning (thinking mode enabled) using the included chat template<\/li>\n<\/ul>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 36.0B parameters (taken from the base model Qwen\/Qwen3.6-35B-A3B)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>24GB (RTX 4090 \/ 3090, etc.)<\/td>\n<td>MLX 4bit<\/td>\n<td>18.2GB<\/td>\n<td>21.8GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><strong>Inference engine support<\/strong> (architecture name matched against each project&#8217;s own model registry in its source code, checked 2026-11-03): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.<\/p>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:runnability --><\/p>\n<h2>Can You Run It Locally?<\/h2>\n<p><strong>Not usable in Ollama, LM Studio and llama.cpp yet \u2014 we have found no GGUF build.<\/strong><\/p>\n<p>The publisher ships MLX \/ safetensors only. However, llama.cpp&#8217;s registry does list this architecture, so <strong>conversion to GGUF is possible<\/strong> and the model will run once someone publishes a converted build. Today it can be run with transformers, vLLM or MLX (Apple Silicon), using the memory figures in the table above.<\/p>\n<p><strong>License \u2014 <code>apache-2.0<\/code> (Commercial use allowed):<\/strong> Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.<\/p>\n<p><strong>Compression:<\/strong> the MLX 4bit build measures 4.35 bits per weight \u2014 about 27% the size of the original 16-bit weights, calculated by this site from the actual file sizes.<\/p>\n<p><em>Compiled by this site&#8217;s code from the published formats, converted builds we have found, and each engine&#8217;s own model registry. &#8220;Not found&#8221; means we have not seen such a build, not that none exists. License summaries are not legal advice \u2014 check the publisher&#8217;s original terms before relying on them.<\/em><\/p>\n<p><!-- \/lmw:runnability --><\/p>\n<p><!-- lmw:lab --><\/p>\n<h2>Our Own Measurements<\/h2>\n<p>Values we measured ourselves on our server (no GPU) by actually reading and running this model&#8217;s files \u2014 not figures copied from the model card. How we measure, and the results for every model: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/observations-en\/\">Our Measurements<\/a>.<\/p>\n<h3>Japanese Token Efficiency<\/h3>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Tokenizer<\/th>\n<th>Tokens per 1,000 Japanese characters<\/th>\n<th>Ratio to the same text in English<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>This model<\/strong><\/td>\n<td><strong>546<\/strong><\/td>\n<td><strong>0.99\u00d7<\/strong><\/td>\n<\/tr>\n<tr>\n<td>Qwen3<\/td>\n<td>688<\/td>\n<td>1.26\u00d7<\/td>\n<\/tr>\n<tr>\n<td>Llama 3.2<\/td>\n<td>744<\/td>\n<td>1.36\u00d7<\/td>\n<\/tr>\n<tr>\n<td>Gemma 3<\/td>\n<td>564<\/td>\n<td>1.03\u00d7<\/td>\n<\/tr>\n<tr>\n<td>gpt-oss<\/td>\n<td>795<\/td>\n<td>1.45\u00d7<\/td>\n<\/tr>\n<tr>\n<td>LLM-jp-3<\/td>\n<td>497<\/td>\n<td>0.85\u00d7<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26\u00d7 as much Japanese fits in the same context length, and generation is faster per character.<\/p>\n<p>Counted with the <code>tokenizer.json<\/code> of <a href=\"https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview\">Edge0\/Edge0-35B-A3B-preview<\/a> on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.<\/p>\n<p><!-- \/lmw:lab --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>15\u201340B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>elyza\/ELYZA-Thinking-1.0-llm-jp-4-32b-a3b<\/td>\n<td>32.1B<\/td>\n<td>80GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/02\/elyza-thinking-1-0-32b-33b-2\/\">ELYZA Releases ELYZA-Thinking-1.0 32B\/33B Reasoning Models<\/a> (2026-10-02)<\/td>\n<\/tr>\n<tr>\n<td>XingChen-AGI\/Xing4.0-29B-A4B<\/td>\n<td>31.2B<\/td>\n<td>24GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/28\/xing4-0-29b-a4b-china-telecom\/\">Xing4.0-29B-A4B Text Generation Model: Our Test Answers, 24GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n<tr>\n<td>Altworld\/Hemmingway-1<\/td>\n<td>26.9B<\/td>\n<td>12GB<\/td>\n<td>cc-by-nc-4.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/28\/hemmingway-1-open-27b-model-specialized-for-human-like-writing\/\">Hemmingway-1 Text Generation Model: 12GB+ VRAM, GGUF Builds<\/a> (2026-09-28)<\/td>\n<\/tr>\n<tr>\n<td>orcarouter\/OrcaSAQ-2-27B<\/td>\n<td>27.8B<\/td>\n<td>16GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/28\/orcasaq2-27b-qwen3-8-27b-quantized\/\">OrcaSAQ-2-27B Text Generation Model: Our Test Answers, 16GB+ VRAM<\/a> (2026-09-28)<\/td>\n<\/tr>\n<tr>\n<td>prism-ml\/Ternary-Bonsai-2-27B-gguf<\/td>\n<td>27.8B<\/td>\n<td>8GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/18\/ternary-bonsai-2-27b-gguf\/\">Ternary-Bonsai-2-27B-gguf: Our Test Answers, 8GB+ VRAM<\/a> (2026-09-18)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<pre><code class=\"language-bash\">pip install -e 'git+https:\/\/github.com\/Edge0-AI\/edge0.git#egg=edge0[fetch]'\n\n# Download this repository into a local directory\nhuggingface-cli download Edge0\/Edge0-35b-a3b-preview --local-dir.\/Edge0-35b-a3b-preview\n\n# Run it\nexport EDGE0_35B_MODEL=$PWD\/Edge0-35b-a3b-preview\nedge0 chat --name edge0-35b --prompt &quot;Introduce yourself&quot;\n\n# Or serve an OpenAI-compatible HTTP API\nedge0 serve --name edge0-35b --port 8085\n<\/code><\/pre>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/24\/edge0-audio8-asr-infinite-2\/\">Audio8-ASR-Infinite Speech Recognition Model: 12GB+ VRAM, File List<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Find models by VRAM<\/strong> (This model runs from the 24GB tier) \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-24gb-en\/\">Other models that run on a 24GB GPU<\/a><\/li>\n<li><strong>Engines that run this model<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-vllm-en\/\">vLLM<\/a><\/li>\n<li><strong>Formats this model is available in<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-mlx-en\/\">MLX format guide and models<\/a><\/li>\n<li><strong>Other models for the same task<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/models-by-task-en\/#task-text\">Other text generation models<\/a><\/li>\n<li><strong>How to read AIME, HumanEval, GPQA Diamond<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-benchmarks-en\/\">Benchmark glossary<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview\">https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview<\/a><\/li>\n<\/ul>\n<p><!-- lmw:updates --><\/p>\n<h2>Update History<\/h2>\n<ul>\n<li>2026-09-24: Rebuilt the article: Added a &#8220;can you run it locally?&#8221; verdict from the distribution formats, converted builds, engine registries, license terms and measured compression<\/li>\n<li>2026-09-25: Changed the title to show what the article covers (VRAM requirements, file list, etc.).<\/li>\n<li>2026-09-28: Added our own measurements: Japanese token efficiency.<\/li>\n<\/ul>\n<p><!-- \/lmw:updates --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Edge0 released Edge0-35B-A3B-preview, a 35B sparse MoE model running on phone-class memory using streaming inference.<\/p>\n","protected":false},"author":1,"featured_media":4153,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[994,977,165,876,996,117],"class_list":["post-543","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-edge0-en","tag-mlx-en","tag-moe-en","tag-on-device-en","tag-qwen3-5-en","tag--en"],"lang":"en","translations":{"en":543,"ja":541},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/543","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=543"}],"version-history":[{"count":71,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/543\/revisions"}],"predecessor-version":[{"id":14825,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/543\/revisions\/14825"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/4153"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=543"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=543"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=543"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}