Edge0-35B-A3B-preview 35B MoE Model for Phone-Class Memory: 24GB+ VRAM

September 11, 2026November 3, 2026

At a Glance

Values determined by this site’s code when the information was collected. Dates are JST.

What we checked ourselves

• It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26× as much Japanese fits in the same context length, and generation is faster per character.

Details and conditions are in “Our Own Measurements” below.

Overview

Edge0 has released “Edge0-35B-A3B-preview", a 35B-class sparse MoE model that runs in smartphone-class memory environments. It is a preview version that operates by integrating a 4-bit quantized model with LoRA and prerouter adapters using the streaming inference framework “edge0" designed for distribution.

Specifications

• Architecture: Qwen3_5MoeForConditionalGeneration (Qwen3_5_MoE)
• Number of parameters: 34.7B
• Number of layers: 40
• Number of experts / Active per token: 256 / 4 (K=4)
• Hidden size: 2048
• License: Apache 2.0
• Framework: edge0 (MLX backend)

Performance

According to measurements by the publishers, the average drop is kept to 3.9 points compared to the fp16 base model Qwen3.5-MoE 35B-A3B. The scores for various benchmarks are as follows.

The measured performance is as follows.

*Measured values for short contexts. KV cache is added for long texts. Expert weights are streamed on-demand from the SSD and are not resident.

From these scores, it can be seen that the quantization loss due to int4 conversion is kept small, maintaining performance levels close to the base model in fields such as competitive mathematics (AIME), coding (HumanEval), difficult science questions (GPQA Diamond), and general knowledge/reasoning (MMLU-Pro). On the other hand, as a limitation of the preview version, optimization for agent-based tasks is not yet advanced.

Strengths and Use Cases

• Edge and on-device inference where GPU VRAM is depleted but fast storage (NVMe or internal flash) is available
• Batch serving on a single host machine (sharing multiple LoRA adapter sets with a single read-only base without re-quantization)
• Multilingual chat and reasoning (thinking mode enabled) using the included chat template

Hardware Requirements

Estimated requirements (calculated by Local Model Watch) — 36.0B parameters (taken from the base model Qwen/Qwen3.6-35B-A3B)

Inference engine support (architecture name matched against each project’s own model registry in its source code, checked 2026-11-03): llama.cpp: registered, vLLM: registered, MLX (mlx-lm): registered.

Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model’s authors. Compare with other models in our VRAM quick reference. What the quantization names mean: glossary.

Can You Run It Locally?

Not usable in Ollama, LM Studio and llama.cpp yet — we have found no GGUF build.

The publisher ships MLX / safetensors only. However, llama.cpp’s registry does list this architecture, so conversion to GGUF is possible and the model will run once someone publishes a converted build. Today it can be run with transformers, vLLM or MLX (Apple Silicon), using the memory figures in the table above.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compression: the MLX 4bit build measures 4.35 bits per weight — about 27% the size of the original 16-bit weights, calculated by this site from the actual file sizes.

Compiled by this site’s code from the published formats, converted builds we have found, and each engine’s own model registry. “Not found" means we have not seen such a build, not that none exists. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Our Own Measurements

Values we measured ourselves on our server (no GPU) by actually reading and running this model’s files — not figures copied from the model card. How we measure, and the results for every model: Our Measurements.

Japanese Token Efficiency

It needs about 21% fewer tokens than the Qwen3 tokenizer for the same Japanese text, so about 1.26× as much Japanese fits in the same context length, and generation is faster per character.

Counted with the tokenizer.json of Edge0/Edge0-35B-A3B-preview on a fixed text we wrote ourselves (876 Japanese characters across news, conversation, technical docs, a formal email, travel writing and a recipe) and its English translation. Fewer tokens mean more Japanese fits in the context window.

Recent Models in the Same Size Class

Models with 15–40B parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site’s estimates; licenses are as stated on the model cards.

How to Get It

pip install -e 'git+https://github.com/Edge0-AI/edge0.git#egg=edge0[fetch]'

# Download this repository into a local directory
huggingface-cli download Edge0/Edge0-35b-a3b-preview --local-dir./Edge0-35b-a3b-preview

# Run it
export EDGE0_35B_MODEL=$PWD/Edge0-35b-a3b-preview
edge0 chat --name edge0-35b --prompt "Introduce yourself"

# Or serve an OpenAI-compatible HTTP API
edge0 serve --name edge0-35b --port 8085

Related Articles

• Audio8-ASR-Infinite Speech Recognition Model: 12GB+ VRAM, File List

What to Read Next

• Find models by VRAM (This model runs from the 24GB tier) → Other models that run on a 24GB GPU
• Engines that run this model → vLLM
• Formats this model is available in → MLX format guide and models
• Other models for the same task → Other text generation models
• How to read AIME, HumanEval, GPQA Diamond → Benchmark glossary

Sources

• https://huggingface.co/Edge0/Edge0-35B-A3B-preview

Update History

• 2026-09-24: Rebuilt the article: Added a “can you run it locally?" verdict from the distribution formats, converted builds, engine registries, license terms and measured compression
• 2026-09-25: Changed the title to show what the article covers (VRAM requirements, file list, etc.).
• 2026-09-28: Added our own measurements: Japanese token efficiency.