Edge0-35B-A3B-Preview: Sparse MoE for Phone-Class Memory
At a Glance
| Item | Value |
|---|---|
| Repository | Edge0/Edge0-35B-A3B-preview |
| Published | 2026-09-08 |
| License | apache-2.0 |
| Formats | MLX / safetensors |
| Paper | arXiv:2609.18063 |
| Source type | Primary source (the publisher itself) |
Values determined by this site’s code at collection time. Dates are JST.
Overview
Edge0 has released “Edge0-35B-A3B-preview", a 35B-class sparse MoE model that runs in smartphone-class memory environments. It is a preview version that operates by integrating a 4-bit quantized model with LoRA and prerouter adapters using the streaming inference framework “edge0" designed for distribution.
Specifications
- Architecture: Qwen3_5MoeForConditionalGeneration (Qwen3_5_MoE)
- Number of parameters: 34.7B
- Number of layers: 40
- Number of experts / Active per token: 256 / 4 (K=4)
- Hidden size: 2048
- License: Apache 2.0
- Framework: edge0 (MLX backend)
Performance
According to measurements by the publishers, the average drop is kept to 3.9 points compared to the fp16 base model Qwen3.5-MoE 35B-A3B. The scores for various benchmarks are as follows.
| Benchmark | edge0-35b (int4) | Qwen3.5-MoE 35B-A3B (fp16) |
|---|---|---|
| AIME 2026 | 86.6 | 92.7 |
| HumanEval | 90.9 | 95.1 |
| GPQA-Diamond | 79.8 | 81.8 |
| MMLU-Pro | 81.0 | 84.6 |
| IFBench | 57.9 | 61.7 |
| Average | 79.2 | 83.2 |
The measured performance is as follows.
| Decode speed | Prefill throughput (cold / warm) | Peak active memory* |
|---|---|---|
| 14.9–17.7 tok/s | 113 / 140 tok/s | 2.9 GiB |
*Measured values for short contexts. KV cache is added for long texts. Expert weights are streamed on-demand from the SSD and are not resident.
From these scores, it can be seen that the quantization loss due to int4 conversion is kept small, maintaining performance levels close to the base model in fields such as competitive mathematics (AIME), coding (HumanEval), difficult science questions (GPQA Diamond), and general knowledge/reasoning (MMLU-Pro). On the other hand, as a limitation of the preview version, optimization for agent-based tasks is not yet advanced.
Strengths and Use Cases
- Edge and on-device inference where GPU VRAM is depleted but fast storage (NVMe or internal flash) is available
- Batch serving on a single host machine (sharing multiple LoRA adapter sets with a single read-only base without re-quantization)
- Multilingual chat and reasoning (thinking mode enabled) using the included chat template
Hardware Requirements
Estimated requirements (calculated by Local Model Watch) — 34.7B parameters
| Your VRAM | Quantization | File size | Est. memory needed |
|---|---|---|---|
| 80GB class (A100 / H100) | U32 | 64.6GB | 77.5GB |
Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-09-18): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.
Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.
Recent Models in the Same Size Class
Models with 15–40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.
→ Scroll horizontally to see all columns
| Model | Parameters | Smallest VRAM tier | License | Our article |
|---|---|---|---|---|
| bartowski/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF | 26.5B | 12GB | apache-2.0 | Pantheon-Reasoning-26B-A4B-1.1-V2 GGUF Quantizations (2026-09-11) |
| nex-agi/Nex-N2.5-mini | 35.1B | 16GB | apache-2.0 | Nex-AGI Releases Open-Weight Long-Task Model Nex-N2.5-mini (2026-09-09) |
How to Get It
pip install -e 'git+https://github.com/Edge0-AI/edge0.git#egg=edge0[fetch]'
# Download this repository into a local directory
huggingface-cli download Edge0/Edge0-35b-a3b-preview --local-dir./Edge0-35b-a3b-preview
# Run it
export EDGE0_35B_MODEL=$PWD/Edge0-35b-a3b-preview
edge0 chat --name edge0-35b --prompt "Introduce yourself"
# Or serve an OpenAI-compatible HTTP API
edge0 serve --name edge0-35b --port 8085