{"id":5509,"date":"2026-09-27T12:45:09","date_gmt":"2026-09-27T03:45:09","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/format-onnx-en\/"},"modified":"2026-09-27T12:45:09","modified_gmt":"2026-09-27T03:45:09","slug":"format-onnx-en","status":"publish","type":"page","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/format-onnx-en\/","title":{"rendered":"ONNX Model Format Explained: Supported Engines and Models"},"content":{"rendered":"<h2>What Is ONNX?<\/h2>\n<p><strong>ONNX<\/strong> (Open Neural Network Exchange) is a common format for moving machine-learning models between tools regardless of the framework that created them (such as PyTorch). A <code>.onnx<\/code> file holds the computation graph together with the weights. Models are usually run with Microsoft&#8217;s <a href=\"https:\/\/onnxruntime.ai\/\">ONNX Runtime<\/a>, and for LLMs there is <a href=\"https:\/\/github.com\/microsoft\/onnxruntime-genai\">ONNX Runtime GenAI<\/a>, which packages the generation loop.<\/p>\n<h2>Why It Matters<\/h2>\n<ul>\n<li><strong>It runs almost anywhere.<\/strong> ONNX Runtime switches between execution providers for CPUs, NVIDIA and AMD GPUs, DirectML on Windows, and the NPUs (dedicated AI processors) in some PCs and phones.<\/li>\n<li><strong>Good for embedding in apps.<\/strong> The same model runs from environments without Python \u2014 C#, C++, Java or the browser. In the browser, Transformers.js uses ONNX models.<\/li>\n<li><strong>Common for small models and audio or vision components.<\/strong> Speech recognition, embedding and image classification models often ship an ONNX version alongside the original.<\/li>\n<\/ul>\n<h2>Tips for Running It Locally<\/h2>\n<ul>\n<li><strong>It is not the mainstream choice for chatting with large LLMs.<\/strong> Ollama and LM Studio do not load ONNX; GGUF and MLX offer far more options for chat.<\/li>\n<li><strong>ONNX files may be built for a specific target.<\/strong> If a repository offers separate files for CPU, GPU (CUDA \/ DirectML) or a particular NPU, pick the one that matches your hardware.<\/li>\n<li><strong>Weights may live in a separate file.<\/strong> Models over 2 GB keep their weights in an external file (such as <code>.onnx_data<\/code>) that must sit next to the <code>.onnx<\/code> file.<\/li>\n<\/ul>\n<p><em>Sources: <a href=\"https:\/\/onnx.ai\/\">the ONNX website<\/a>, <a href=\"https:\/\/onnxruntime.ai\/docs\/\">ONNX Runtime documentation<\/a>, <a href=\"https:\/\/github.com\/microsoft\/onnxruntime-genai\">the microsoft\/onnxruntime-genai README<\/a> and <a href=\"https:\/\/huggingface.co\/docs\/transformers.js\/index\">Transformers.js documentation<\/a> (all as of 2026-09-27).<\/em><\/p>\n<h2>Our Coverage and Data<\/h2>\n<p>Local Model Watch has published 1 article(s) on models available in ONNX: 1 where the repository itself is in ONNX, and 0 where we found a ONNX build of the model. The lists below only include builds we have checked (the publisher&#8217;s organization and well-known quantizers); a model missing here may still have a ONNX build elsewhere. Part of our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/formats-en\/\">model format index<\/a>.<\/p>\n<h2>Models Available in ONNX<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Published<\/th>\n<th>Model<\/th>\n<th>Where to get it<\/th>\n<th>Quantizations<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>Article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>2026-09-19<\/td>\n<td>inclusionAI\/Realtime-Venus<\/td>\n<td>This repository<\/td>\n<td>\u2014<\/td>\n<td>24GB<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/19\/realtime-venus-multimodal-conversational-ai\/\">Realtime-Venus Full-Duplex Audio-Video Dialog Model: 24GB+ VRAM<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>&#8220;Smallest VRAM tier&#8221; is the smallest tier in the requirements table of each article (for other builds, of those builds; for image, video and audio models, always the article&#8217;s own table, which counts every component). Leave headroom for context length.<\/em><\/p>\n<p><em>Last updated 2026-09-19 (JST). The explanation at the top of this page was written with the help of AI from the primary sources it cites. The tables under &#8220;Our Coverage and Data&#8221; are assembled by code from our article log.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>What Is ONNX? ONNX (Open Neural Network Exchange) is a common format for moving machine-learning models between tools regardless of the framework that created them (such as PyTorch). A .onnx file holds the computation graph together with the weights. Models are usually run with Microsoft&#8217;s ONNX Runtime, and for LLMs there is ONNX Runtime GenAI, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-5509","page","type-page","status-publish","hentry"],"lang":"en","translations":{"en":5509,"ja":5508},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages\/5509","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=5509"}],"version-history":[{"count":0,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/pages\/5509\/revisions"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=5509"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}