{"id":1038,"date":"2026-09-17T03:08:29","date_gmt":"2026-09-16T18:08:29","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/17\/qwen-25-1b-rlcd-mlx-constrained-decoding\/"},"modified":"2026-09-18T21:42:05","modified_gmt":"2026-09-18T12:42:05","slug":"qwen-25-1b-rlcd-mlx-constrained-decoding","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/17\/qwen-25-1b-rlcd-mlx-constrained-decoding\/","title":{"rendered":"Fast Structured Generation on Apple Silicon with MLX and Qwen"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/harshatheg\/Qwen-2.5-1B-RLCD\">harshatheg\/Qwen-2.5-1B-RLCD<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-16<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td>MLX<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code at collection time. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>An implementation and model for the MLX framework, <code>harshatheg\/Qwen-2.5-1B-RLCD<\/code>, has been released to achieve high-efficiency inference in Apple Silicon environments. This repository adopts <code>Qwen\/Qwen2.5-1.5B-Instruct<\/code> as its base model and provides an inference engine incorporating parallel constrained decoding.<\/p>\n<p>Traditional autoregressive decoding generates tokens one by one in sequence, which causes latency to increase as schemas grow larger. By evaluating multi-field JSON schemas and choices (enum, boolean) in parallel, this method achieves a 5.6x to 7.0x latency reduction and 100% schema validity on an Apple Silicon M4 Max.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Base Model: Qwen\/Qwen2.5-1.5B-Instruct<\/li>\n<li>Parameter Count: 1.54B (Non-embedding parameters: 1.31B)<\/li>\n<li>Context Length: Full 32,768 tokens (up to 8,192 generation tokens)<\/li>\n<li>License: apache-2.0<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>According to measurements published by the author using <code>mlx-community\/Qwen2.5-1.5B-Instruct-4bit<\/code> on an Apple Silicon M4 Max (macOS Sequoia) environment, the comparison between the standard autoregressive baseline and parallel constrained decoding is as follows:<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th style=\"text-align: left;\">Scenario<\/th>\n<th style=\"text-align: left;\">Fields<\/th>\n<th style=\"text-align: left;\">Autoregressive Baseline<\/th>\n<th style=\"text-align: left;\">Parallel Constrained<\/th>\n<th style=\"text-align: left;\">Latency Speedup<\/th>\n<th style=\"text-align: left;\">Syntax Validity<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: left;\">Fintech Fraud Routing<\/td>\n<td style=\"text-align: left;\">4 fields<\/td>\n<td style=\"text-align: left;\">420 ms (120 tok\/s)<\/td>\n<td style=\"text-align: left;\">75 ms<\/td>\n<td style=\"text-align: left;\">5.6x<\/td>\n<td style=\"text-align: left;\">100% guaranteed<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">Code Security Audit<\/td>\n<td style=\"text-align: left;\">4 fields<\/td>\n<td style=\"text-align: left;\">380 ms (125 tok\/s)<\/td>\n<td style=\"text-align: left;\">68 ms<\/td>\n<td style=\"text-align: left;\">5.6x<\/td>\n<td style=\"text-align: left;\">100% guaranteed<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">High-Cardinality Tariff<\/td>\n<td style=\"text-align: left;\">1 field (255 choices)<\/td>\n<td style=\"text-align: left;\">500 ms (118 tok\/s)<\/td>\n<td style=\"text-align: left;\">89 ms<\/td>\n<td style=\"text-align: left;\">5.6x<\/td>\n<td style=\"text-align: left;\">100% guaranteed<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: left;\">Enterprise Support Triage<\/td>\n<td style=\"text-align: left;\">28 fields<\/td>\n<td style=\"text-align: left;\">1,900 ms (130 tok\/s)<\/td>\n<td style=\"text-align: left;\">270 ms<\/td>\n<td style=\"text-align: left;\">7.0x<\/td>\n<td style=\"text-align: left;\">100% guaranteed<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>The author&#8217;s measurements show that the effect of parallel processing becomes more pronounced in scenarios with an increasing number of target fields. For &#8220;Enterprise Support Triage,&#8221; which extracts 28 fields, conventional autoregressive generation required 1,900 ms, whereas this method achieved a significant speedup of 7.0x at 270 ms.<\/p>\n<p>Additionally, consistent speedups of 5.6x (75 ms to 89 ms) were recorded for the high-cardinality &#8220;High-Cardinality Tariff&#8221; with 255 choices, as well as the 4-field &#8220;Fintech Fraud Routing&#8221; and &#8220;Code Security Audit.&#8221; In all test cases, syntax validity is 100% guaranteed, and the strength lies in executing high-speed processing while completely preventing grammar errors and missing fields during structured data output.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>Building upon the high instruction-following and structured data comprehension capabilities of the base model <code>Qwen\/Qwen2.5-1.5B-Instruct<\/code>, this combines parallel constrained decoding technology to exhibit specialized strengths in information extraction and classification tasks.<\/p>\n<p>Specifically, input context and schema definition text are preloaded (prefilled) into the MLX KV Cache (Key-Value Cache) just once, and that state is broadcast in parallel to the evaluation of each field. By applying mask extraction (logit slicing) only to token IDs permitted for each field to directly calculate probabilities from predefined choices (booleans or up to 255 enums), unnecessary token generation loops are avoided.<\/p>\n<p>This makes it highly effective in business scenarios such as:<br \/>\n&#8211; Fintech fraud routing (identifying risk levels and determining the necessity of confirmation)<br \/>\n&#8211; Security audits (classifying vulnerable code and assigning remediation priorities)<br \/>\n&#8211; Automated triage of large-scale customer support tickets (department routing and urgency setting)<br \/>\n&#8211; Identifying product codes or tariff classifications (such as HS codes) with numerous classification candidates<\/p>\n<p>Since JSON syntax breaks or unnecessary key outputs (hallucinations) do not occur by design, it is extremely well-suited for integration with backend systems and the construction of automated pipelines where accuracy is required.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong> \u2014 1.5B parameters (taken from the base model Qwen\/Qwen2.5-1.5B-Instruct)<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>8GB (RTX 4060 \/ 3060 Ti, etc.)<\/td>\n<td>BF16<\/td>\n<td>2.9GB<\/td>\n<td>3.5GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<p><!-- lmw:peers --><\/p>\n<h2>Recent Models in the Same Size Class<\/h2>\n<p><em>Models with <\/em><em>up to 4B<\/em><em> parameters that Local Model Watch covered recently, listed by code from our article log for comparison. VRAM tiers are this site&#8217;s estimates; licenses are as stated on the model cards.<\/em><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model<\/th>\n<th>Parameters<\/th>\n<th>Smallest VRAM tier<\/th>\n<th>License<\/th>\n<th>Our article<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>tencent\/Simple-Attention-Sparsification<\/td>\n<td>4.0B<\/td>\n<td>12GB<\/td>\n<td>\u2014<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/14\/tencent-simple-attention-sparsification-qwen3\/\">Tencent Releases Simple Attention Sparsification for Qwen3<\/a> (2026-09-14)<\/td>\n<\/tr>\n<tr>\n<td>openbmb\/MiniCPM5-2B<\/td>\n<td>2.5B<\/td>\n<td>4GB<\/td>\n<td>apache-2.0<\/td>\n<td><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/08\/minicpm5-2b-released\/\">OpenBMB Releases MiniCPM5-2B: A SOTA 2B On-Device Model<\/a> (2026-09-07)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><!-- \/lmw:peers --><\/p>\n<h2>How to Get It<\/h2>\n<p>This model and inference engine are available through the MLX library. Setup procedures and dependency installations can be executed with the following commands:<\/p>\n<pre><code class=\"language-bash\">git clone https:\/\/github.com\/your-org\/parallel-constrained-decoding.git\ncd parallel-constrained-decoding\n\npython3 -m venv.venv\nsource.venv\/bin\/activate\npip install -r requirements.txt\n<\/code><\/pre>\n<p>From Python code, you can use <code>StructuredSchema<\/code> to define schemas and call the <code>run_parallel_generation<\/code> function to execute parallel high-speed inference. You can also launch the web interface (<code>python3 -m uvicorn server.app:app --host 0.0.0.0 --port 8000<\/code>) to check latency comparisons with conventional autoregressive generation and structured data output right in your browser.<\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/huggingface.co\/harshatheg\/Qwen-2.5-1B-RLCD\">https:\/\/huggingface.co\/harshatheg\/Qwen-2.5-1B-RLCD<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Explore harshatheg\/Qwen-2.5-1B-RLCD, an MLX implementation using parallel constrained decoding on Apple Silicon for high-speed, structured JSON output.<\/p>\n","protected":false},"author":1,"featured_media":1037,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[505,1386,977,1388,1390,117],"class_list":["post-1038","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-apple-silicon-en","tag-harshatheg-qwen-2-5-1b-rlcd-en","tag-mlx-en","tag-qwen2-5-en","tag--en"],"lang":"en","translations":{"en":1038,"ja":1036},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1038","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=1038"}],"version-history":[{"count":5,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1038\/revisions"}],"predecessor-version":[{"id":1762,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1038\/revisions\/1762"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/1037"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=1038"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=1038"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=1038"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}