ONNX Model Format Explained: Supported Engines and Models
What Is ONNX?
ONNX (Open Neural Network Exchange) is a common format for moving machine-learning models between tools regardless of the framework that created them (such as PyTorch). A .onnx file holds the computation graph together with the weights. Models are usually run with Microsoft’s ONNX Runtime, and for LLMs there is ONNX Runtime GenAI, which packages the generation loop.
Why It Matters
- It runs almost anywhere. ONNX Runtime switches between execution providers for CPUs, NVIDIA and AMD GPUs, DirectML on Windows, and the NPUs (dedicated AI processors) in some PCs and phones.
- Good for embedding in apps. The same model runs from environments without Python — C#, C++, Java or the browser. In the browser, Transformers.js uses ONNX models.
- Common for small models and audio or vision components. Speech recognition, embedding and image classification models often ship an ONNX version alongside the original.
Tips for Running It Locally
- It is not the mainstream choice for chatting with large LLMs. Ollama and LM Studio do not load ONNX; GGUF and MLX offer far more options for chat.
- ONNX files may be built for a specific target. If a repository offers separate files for CPU, GPU (CUDA / DirectML) or a particular NPU, pick the one that matches your hardware.
- Weights may live in a separate file. Models over 2 GB keep their weights in an external file (such as
.onnx_data) that must sit next to the.onnxfile.
Sources: the ONNX website, ONNX Runtime documentation, the microsoft/onnxruntime-genai README and Transformers.js documentation (all as of 2026-09-27).
Our Coverage and Data
Local Model Watch has published 1 article(s) on models available in ONNX: 1 where the repository itself is in ONNX, and 0 where we found a ONNX build of the model. The lists below only include builds we have checked (the publisher’s organization and well-known quantizers); a model missing here may still have a ONNX build elsewhere. Part of our model format index.
Models Available in ONNX
→ Scroll horizontally to see all columns
| Published | Model | Where to get it | Quantizations | Smallest VRAM tier | Article |
|---|---|---|---|---|---|
| 2026-09-19 | inclusionAI/Realtime-Venus | This repository | — | 24GB | Realtime-Venus Full-Duplex Audio-Video Dialog Model: 24GB+ VRAM |
“Smallest VRAM tier" is the smallest tier in the requirements table of each article (for other builds, of those builds; for image, video and audio models, always the article’s own table, which counts every component). Leave headroom for context length.
Last updated 2026-09-19 (JST). The explanation at the top of this page was written with the help of AI from the primary sources it cites. The tables under “Our Coverage and Data" are assembled by code from our article log.