{"id":543,"date":"2026-09-11T12:14:06","date_gmt":"2026-09-11T03:14:06","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/11\/edge0-35b-a3b-preview-sparse-moe\/"},"modified":"2026-09-18T21:41:59","modified_gmt":"2026-09-18T12:41:59","slug":"edge0-35b-a3b-preview-sparse-moe","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/11\/edge0-35b-a3b-preview-sparse-moe\/","title":{"rendered":"Edge0-35B-A3B-Preview: Sparse MoE for Phone-Class Memory"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview\">Edge0\/Edge0-35B-A3B-preview<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-08<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td>MLX \/ safetensors<\/td>\n<\/tr>\n<tr>\n<td>Paper<\/td>\n<td><a href=\"https:\/\/arxiv.org\/abs\/2609.18063\">arXiv:2609.18063<\/a><\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code at collection time. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>Edge0 has released &#8220;Edge0-35B-A3B-preview&#8221;, a 35B-class sparse MoE model that runs in smartphone-class memory environments. It is a preview version that operates by integrating a 4-bit quantized model with LoRA and prerouter adapters using the streaming inference framework &#8220;edge0&#8221; designed for distribution.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Architecture: Qwen3_5MoeForConditionalGeneration (Qwen3_5_MoE)<\/li>\n<li>Number of parameters: 34.7B<\/li>\n<li>Number of layers: 40<\/li>\n<li>Number of experts \/ Active per token: 256 \/ 4 (K=4)<\/li>\n<li>Hidden size: 2048<\/li>\n<li>License: Apache 2.0<\/li>\n<li>Framework: edge0 (MLX backend)<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>According to measurements by the publishers, the average drop is kept to 3.9 points compared to the fp16 base model Qwen3.5-MoE 35B-A3B. The scores for various benchmarks are as follows.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Benchmark<\/th>\n<th style=\"text-align: right;\">edge0-35b (int4)<\/th>\n<th style=\"text-align: right;\">Qwen3.5-MoE 35B-A3B (fp16)<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>AIME 2026<\/td>\n<td style=\"text-align: right;\">86.6<\/td>\n<td style=\"text-align: right;\">92.7<\/td>\n<\/tr>\n<tr>\n<td>HumanEval<\/td>\n<td style=\"text-align: right;\">90.9<\/td>\n<td style=\"text-align: right;\">95.1<\/td>\n<\/tr>\n<tr>\n<td>GPQA-Diamond<\/td>\n<td style=\"text-align: right;\">79.8<\/td>\n<td style=\"text-align: right;\">81.8<\/td>\n<\/tr>\n<tr>\n<td>MMLU-Pro<\/td>\n<td style=\"text-align: right;\">81.0<\/td>\n<td style=\"text-align: right;\">84.6<\/td>\n<\/tr>\n<tr>\n<td>IFBench<\/td>\n<td style=\"text-align: right;\">57.9<\/td>\n<td style=\"text-align: right;\">61.7<\/td>\n<\/tr>\n<tr>\n<td><strong>Average<\/strong><\/td>\n<td style=\"text-align: right;\"><strong>79.2<\/strong><\/td>\n<td style=\"text-align: right;\"><strong>83.2<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>The measured performance is as follows.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Decode speed<\/th>\n<th>Prefill throughput (cold \/ warm)<\/th>\n<th>Peak active memory*<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>14.9\u201317.7 tok\/s<\/td>\n<td>113 \/ 140 tok\/s<\/td>\n<td>2.9 GiB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>*Measured values for short contexts. KV cache is added for long texts. Expert weights are streamed on-demand from the SSD and are not resident.<\/p>\n<p>From these scores, it can be seen that the quantization loss due to int4 conversion is kept small, maintaining performance levels close to the base model in fields such as competitive mathematics (AIME), coding (HumanEval), difficult science questions (GPQA Diamond), and general knowledge\/reasoning (MMLU-Pro). On the other hand, as a limitation of the preview version, optimization for agent-based tasks is not yet advanced.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<ul>\n<li>Edge and on-device inference where GPU VRAM is depleted but fast storage (NVMe or internal flash) is available<\/li>\n<li>Batch serving on a single host machine (sharing multiple LoRA adapter sets with a single read-only base without re-quantization)<\/li>\n<li>Multilingual chat and reasoning (thinking mode enabled) using the included chat template<\/li>\n<\/ul>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 34.7B parameters<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>80GB class (A100 \/ H100)<\/td>\n<td>U32<\/td>\n<td>64.6GB<\/td>\n<td>77.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><strong>Inference engine support<\/strong> (architecture name matched against each project&#8217;s own model registry in its source code, checked 2026-09-18): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.<\/p>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>15\u201340B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>bartowski\/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF<\/td>\n<td>26.5B<\/td>\n<td>12GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/11\/pantheon-reasoning-26b-gguf\/\">Pantheon-Reasoning-26B-A4B-1.1-V2 GGUF Quantizations<\/a> (2026-09-11)<\/td>\n<\/tr>\n<tr>\n<td>nex-agi\/Nex-N2.5-mini<\/td>\n<td>35.1B<\/td>\n<td>16GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/09\/nex-n25-mini-released\/\">Nex-AGI Releases Open-Weight Long-Task Model Nex-N2.5-mini<\/a> (2026-09-09)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<pre><code class=\"language-bash\">pip install -e 'git+https:\/\/github.com\/Edge0-AI\/edge0.git#egg=edge0[fetch]'\n\n# Download this repository into a local directory\nhuggingface-cli download Edge0\/Edge0-35b-a3b-preview --local-dir.\/Edge0-35b-a3b-preview\n\n# Run it\nexport EDGE0_35B_MODEL=$PWD\/Edge0-35b-a3b-preview\nedge0 chat --name edge0-35b --prompt &quot;Introduce yourself&quot;\n\n# Or serve an OpenAI-compatible HTTP API\nedge0 serve --name edge0-35b --port 8085\n<\/code><\/pre>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview\">https:\/\/huggingface.co\/Edge0\/Edge0-35B-A3B-preview<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Edge0 released Edge0-35B-A3B-preview, a 35B sparse MoE model running on phone-class memory using streaming inference.<\/p>\n","protected":false},"author":1,"featured_media":542,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[994,977,165,876,996,117],"class_list":["post-543","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-edge0-en","tag-mlx-en","tag-moe-en","tag-on-device-en","tag-qwen3-5-en","tag--en"],"lang":"en","translations":{"en":543,"ja":541},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/543","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=543"}],"version-history":[{"count":7,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/543\/revisions"}],"predecessor-version":[{"id":1750,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/543\/revisions\/1750"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/542"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=543"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=543"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=543"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}