{"id":1821,"date":"2026-09-19T06:50:49","date_gmt":"2026-09-18T21:50:49","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/09\/19\/cactus-compute-needle-3\/"},"modified":"2026-09-19T06:55:02","modified_gmt":"2026-09-18T21:55:02","slug":"cactus-compute-needle-3","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/09\/19\/cactus-compute-needle-3\/","title":{"rendered":"Cactus Compute Releases Needle 3: 8\u201329 MB On-Device Automation Model"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Repository<\/td>\n<td><a href=\"https:\/\/huggingface.co\/Cactus-Compute\/needle3\">Cactus-Compute\/needle3<\/a><\/td>\n<\/tr>\n<tr>\n<td>Published<\/td>\n<td>2026-09-16<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Formats<\/td>\n<td>safetensors<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code at collection time. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<h2>Overview<\/h2>\n<p>Cactus Compute has released <strong>Needle 3<\/strong> as an &#8220;automation foundation model&#8221; designed for smartphones, wearables, smart home devices, robots, automotive systems, and microcontrollers. According to the publishers, the entire model fits into a single 8 to 29 MB file using 2bit quantization (CQ2). While intentionally trading away general-purpose chat capabilities, it reportedly outperforms 10x larger models in mobile tool calling and matches 2-3x larger models in structured extraction. The &#8220;Show HN&#8221; post on Hacker News has gathered a score of 128 with 67 comments at the time of collection.<\/p>\n<p>Needle 3 narrows on-device operations down to three core tasks: <strong>tool calling<\/strong>, which selects the appropriate function from those exposed by an app and populates its arguments; <strong>structured extraction<\/strong>, which extracts typed fields from unstructured text according to a declared schema (also applicable to classification); and <strong>text embedding<\/strong>, returning sentence vectors using the same model. The model card emphasizes that if given two requests, it returns two sequential calls, and for requests that do not fit any tool, it returns an empty list rather than guessing.<\/p>\n<h2>Specifications<\/h2>\n<ul>\n<li>Parameter count: 29M to 121M (a &#8220;ladder&#8221; structure ranging from 2 to 20 layers. 121M for 20 layers, 98M for 16 layers, 52M for 8 layers, and 29M for 4 layers)<\/li>\n<li>Architecture: Laddered Simple Attention Network (not a Transformer). Consists of Monarch Hadamard MLP instead of FFN, GQA attention with causal convolution taps, engram n-gram memory read via gather, and multi-lane hyper-connections. Since the majority of parameters reside on the engram side, the computational cost for the 121M model is described as equivalent to a 50M model<\/li>\n<li>&#8220;Intelligence laddering&#8221;: Trained to function as standalone sub-networks when cut at any depth from 2 to 20 layers, allowing sizes to be selected from 2L to 20L depending on the use case<\/li>\n<li>Quantization: CQ2-bit by Cactus Quants (2.125 bits per weight according to the publisher&#8217;s blog). Distributed in a proprietary <code>.cact<\/code> format<\/li>\n<li>Output constraints: Constrains all tokens during decoding using a byte-level grammar compiled from the schema, ensuring the output JSON is always parsable. Each response includes calibrated confidence scores generated by a trained head<\/li>\n<li>Training data: 360B tokens of proprietary structured datasets<\/li>\n<li>Speed (measured by publishers): 400 to 4k tokens\/s for decoding and 1 to 10k tokens\/s for prefill on a Raspberry Pi 5. MFLOPs per token are less than half compared to a Transformer of the same configuration<\/li>\n<\/ul>\n<h2>Performance<\/h2>\n<p>The model card includes charts for six benchmark results. It notes that Needle 3 is measured with confidence gating enabled in the distributed CQ2 binary, comparison open models are measured on vLLM at f16, and DeepSeek V4 Flash is measured via cloud API. The figures from the charts are transcribed into the table below (the three tool-calling benchmarks are in accuracy %, and the three extraction benchmarks are in field-level F1 %. Blank cells indicate combinations not present in the charts).<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Model (Scale)<\/th>\n<th>Mobile Actions<\/th>\n<th>DroidCall<\/th>\n<th>BFCL v4<\/th>\n<th>DSTC8<\/th>\n<th>SNIPS gold<\/th>\n<th>SNIPS 7-way<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>DeepSeek V4 Flash(cloud)<\/td>\n<td>88.4<\/td>\n<td>60.5<\/td>\n<td>77.2<\/td>\n<td>80.0<\/td>\n<td>69.4<\/td>\n<td>66.7<\/td>\n<\/tr>\n<tr>\n<td>Needle3-20L-121M<\/td>\n<td>86.0<\/td>\n<td>47.0<\/td>\n<td>50.2<\/td>\n<td>40.7<\/td>\n<td>30.2<\/td>\n<td>24.7<\/td>\n<\/tr>\n<tr>\n<td>Needle3-16L-98M<\/td>\n<td>80.7<\/td>\n<td>40.0<\/td>\n<td>41.3<\/td>\n<td>28.5<\/td>\n<td>23.5<\/td>\n<td>19.2<\/td>\n<\/tr>\n<tr>\n<td>Needle3-8L-52M<\/td>\n<td>36.8<\/td>\n<td>36.5<\/td>\n<td>28.2<\/td>\n<td>15.3<\/td>\n<td>16.6<\/td>\n<td>10.1<\/td>\n<\/tr>\n<tr>\n<td>Needle3-4L-29M<\/td>\n<td>11.7<\/td>\n<td>21.0<\/td>\n<td>19.5<\/td>\n<td>6.9<\/td>\n<td>7.7<\/td>\n<td>4.3<\/td>\n<\/tr>\n<tr>\n<td>LFM2.5 1.2B<\/td>\n<td>82.4<\/td>\n<td>35.5<\/td>\n<td>62.0<\/td>\n<td>48.0<\/td>\n<td>43.0<\/td>\n<td>38.0<\/td>\n<\/tr>\n<tr>\n<td>LFM2.5 350M<\/td>\n<td>72.8<\/td>\n<td>32.5<\/td>\n<td>59.1<\/td>\n<td>20.0<\/td>\n<td>34.0<\/td>\n<td>29.0<\/td>\n<\/tr>\n<tr>\n<td>LFM2.5 230M<\/td>\n<td>69.3<\/td>\n<td>11.5<\/td>\n<td>46.3<\/td>\n<td>53.0<\/td>\n<td>27.0<\/td>\n<td>22.0<\/td>\n<\/tr>\n<tr>\n<td>Qwen3.5 0.8B<\/td>\n<td>76.0<\/td>\n<td>28.0<\/td>\n<td>56.8<\/td>\n<td>49.0<\/td>\n<td>35.0<\/td>\n<td>34.0<\/td>\n<\/tr>\n<tr>\n<td>FunctionGemma 270M<\/td>\n<td>65.1<\/td>\n<td>16.5<\/td>\n<td>46.6<\/td>\n<td>27.0<\/td>\n<td>29.0<\/td>\n<td>14.0<\/td>\n<\/tr>\n<tr>\n<td>Needle 2(45M)<\/td>\n<td>63.5<\/td>\n<td>17.0<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Apple FM(3.0B)<\/td>\n<td>57.6<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>According to the charts in the model card, on Mobile Actions (961 exact-match instances converting smartphone commands into Android intents), the 20-layer Needle 3 scores 86.0, coming within 2.4 points of the cloud-based DeepSeek V4 Flash (88.4) and outperforming the 10x larger LFM2.5 1.2B (82.4). On DroidCall (200 instances requiring sequential return of two calls), it scores 47.0, ranking second behind DeepSeek V4 Flash and outperforming LFM2.5 1.2B (35.5) and Qwen3.5 0.8B (28.0).<\/p>\n<p>On the other hand, on BFCL v4 (a metric measuring general function calling accuracy in tool selection and argument assembly), it achieves only 50.2, falling behind LFM2.5 1.2B (62.0), 350M (59.1), and Qwen3.5 0.8B (56.8). Extraction benchmarks such as DSTC8, SNIPS gold, and SNIPS 7-way show similar results, scoring lower than LFM2.5 1.2B and Qwen3.5 0.8B, with a 30 to 40 point gap compared to DeepSeek V4 Flash. While the model card&#8217;s claim of being &#8220;specialized and strong in mobile-oriented tool calling&#8221; holds true per the table, the claim of matching &#8220;2-3x larger models&#8221; is not clearly supported by the charts for extraction, where performance wins and losses vary by domain compared to LFM2.5 230M and 350M.<\/p>\n<p>Furthermore, the post title stating it &#8220;rivals DeepSeek V4 Flash&#8221; refers to <strong>after fine-tuning<\/strong>, not out-of-the-box performance. According to the model card, fine-tuning on DroidCall improves all sub-networks by 18 to 36 points, allowing sub-networks with 4 layers (29M) or more to surpass DeepSeek V4 Flash on downstream tasks. Conversely, smaller sub-networks of 8 and 4 layers do not reach practical levels out of the box on Mobile Actions (scoring 36.8 and 11.7), meaning the model is designed to be fine-tuned on custom tool definitions before use. Note that these figures are published by the creators themselves and have not been independently verified.<\/p>\n<h2>Strengths and Use Cases<\/h2>\n<p>Tagged with <code>tool-calling<\/code>, <code>function-calling<\/code>, <code>on-device<\/code>, <code>edge<\/code>, and <code>webassembly<\/code>, the model&#8217;s use cases are strictly limited to &#8220;on-device automation&#8221; as specified by the model card. Specific examples cited by the publishers include smart homes (translating &#8220;dim the bedroom lights and lock up&#8221; into two calls executed offline without cloud roundtrips), robot vacuum instructions like &#8220;clean the kitchen, skip the bedroom&#8221;, smartphone operations such as creating photo albums or dimming the screen, processing credit card notification cards into merchant name, amount, and date on smartwatches, navigation and local search on AR glasses, climate, media, and navigation control in cars, and drafting emails or starting timers on PCs. All of these are tasks that translate spoken language into predefined function calls.<\/p>\n<p>General chat and text generation are excluded from the start, and the publishers explicitly state that they are &#8220;traded away in exchange for general chat capability.&#8221; Because the model reads tool descriptions literally, accuracy depends heavily on &#8220;tool design practices&#8221; such as assigning one tool per action, naming enum values using terms users actually speak, giving default values to required arguments, and attaching regex <code>triggers<\/code> to intents that must route to specific tools. Calls with a confidence score below 0.1 are relegated to <code>suppressed_calls<\/code> by the engine, while how to handle scores above that threshold (immediate execution, confirmation, or rejection) is left to the application.<\/p>\n<h2>How It Varies from Similar Models<\/h2>\n<p>(Note: This section heading corresponds to &#8216;\u985e\u4f3c\u30e2\u30c7\u30eb\u3068\u306e\u9055\u3044&#8217; in the Japanese text, maintained as required.)<\/p>\n<h2>Hacker News Reaction<\/h2>\n<p>Discussion heavily featured reports from users testing the official browser demo. While direct commands like &#8220;turn on\/off all room lights&#8221; or &#8220;the bathroom is dark&#8221; worked, multiple users reported malfunctions with indirect phrasing, such as &#8220;warm up the house&#8221; triggering a command to change lights to a warm color, &#8220;it&#8217;s too freezing&#8221; lowering the thermostat, and &#8220;the cat threw up near the fridge&#8221; sending the robot vacuum to clean the living room. Some pointed out that incorrect responses had low confidence scores, prompting suggestions that the demo should implement a threshold.<\/p>\n<p>Other users shared comparative results from their own tasks. For tool calling on a game database, FunctionGemma fine-tuned in BF16 achieved correct tool shapes of 209\/230 (90.9%) and exact argument matches of 196\/230 (85.2%), whereas Needle 3 fine-tuned in W4A8 reached only 74\/230 (32.2%) and 47\/230 (20.4%). Another report noted that MNLI performed better for labeling use cases. Comments included remarks like &#8220;you have to match the use case quite precisely&#8221; and &#8220;now that claims of small models beating LLMs are increasing, I wish they would clearly state what they cannot do.&#8221; On the other hand, development-focused empathy emerged regarding its multi-platform support, the appreciation for the ladder architecture, and the conviction that dataset quality is decisive.<\/p>\n<p><!-- lmw:hardware --><\/p>\n<h2>Hardware Requirements<\/h2>\n<p><strong>Estimated requirements (calculated by Local Model Watch)<\/strong><\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Your VRAM<\/th>\n<th>Quantization<\/th>\n<th>File size<\/th>\n<th>Est. memory needed<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>4GB (laptop iGPU \/ phone class)<\/td>\n<td>\u305d\u306e\u307e\u307e\u306e\u7cbe\u5ea6<\/td>\n<td>0.2GB<\/td>\n<td>0.3GB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><strong>Inference engine support<\/strong> (architecture name matched against each project&#8217;s own model registry in its source code, checked 2026-09-18): llama.cpp: not registered, vLLM: not registered, MLX (mlx-lm): not registered. &#8220;Not registered&#8221; means the name is absent from that registry today, not that the model cannot run.<\/p>\n<p><em>Memory estimates add a 20% runtime overhead (KV cache, etc.) to the actual size of the distributed files. Actual usage varies with context length, batch size and inference engine. These figures are computed by this site from file sizes, not published by the model&#8217;s authors. Compare with other models in our <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/vram-guide-en\/\">VRAM quick reference<\/a>. What the quantization names mean: <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/glossary-quantization-en\/\">glossary<\/a>.<\/em><\/p>\n<p><!-- \/lmw:hardware --><\/p>\n<h2>How to Get It<\/h2>\n<ul>\n<li>Distribution format: Hugging Face <code>Cactus-Compute\/needle3<\/code> hosts the 20-layer <code>needle3.cact<\/code>, fine-tuning <code>needle3.safetensors<\/code> checkpoints, and platform-specific inference engines (each under 1 MB)<\/li>\n<li>Python package: <code>pip install cactus-needle<\/code>. Decorate functions with <code>@needle.tool<\/code>, and use <code>needle.Needle(tools=[...]).run(\"...\")<\/code> to handle everything from tool selection to execution and result return. Extraction can be called by passing Pydantic models like <code>needle.extract(text, Invoice)<\/code>. The engine and weights are fetched from Hugging Face and cached on first run<\/li>\n<li>Supported engines: Runs on a dedicated engine rather than llama.cpp or Ollama. Targets include macOS (Apple Silicon), Linux (x86_64 \/ arm64 \/ armv7 \/ riscv64 \/ mipsel), Windows (x64 \/ ARM), Android, iOS, tvOS \/ watchOS, browsers (wasm), and WASI components. Export engines and weights at any depth using <code>needle build --platform &lt;folder_name&gt; --layers N<\/code>, and run via CLI (<code>.\/needle --model needle3.cact --tools tools.json --prompt \"...\"<\/code>) or <code>--serve<\/code>. A C API (<code>libneedle.a<\/code> \/ <code>needle.h<\/code>) is also bundled<\/li>\n<li>Fine-tuning: Train LoRA on the 20-layer frozen base using <code>needle finetune data.jsonl --epochs 10 --out adapter.safetensors<\/code>, then merge adapters with <code>needle build --lora adapter.safetensors --out tuned.cact<\/code> (specifying depth via <code>--layers N<\/code>) to export sub-networks at any depth as 4bit <code>.cact<\/code> files. Post-training and quantization at the same 2bit level as the distributed models are provided exclusively through the Cactus Platform<\/li>\n<li>The aforementioned &#8220;Hardware Requirements&#8221; table (in the source notes) was calculated by our site&#8217;s code from the actual sizes of the repository&#8217;s <code>safetensors<\/code> checkpoints (base precision for fine-tuning), and the <code>.cact<\/code> files actually distributed and executed are smaller<\/li>\n<\/ul>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/cactuscompute.com\/needle\">https:\/\/cactuscompute.com\/needle<\/a><\/li>\n<li><a href=\"https:\/\/huggingface.co\/Cactus-Compute\/needle3\">https:\/\/huggingface.co\/Cactus-Compute\/needle3<\/a><\/li>\n<li><a href=\"https:\/\/news.ycombinator.com\/item?id=49748553\">https:\/\/news.ycombinator.com\/item?id=49748553<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Cactus Compute has released Needle 3, a tiny 2bit automation foundation model for mobile, edge, and IoT devices focusing on tool calling and extraction.<\/p>\n","protected":false},"author":1,"featured_media":1843,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[310],"tags":[1605,1607,1609,1611,1613,1547,358,1555],"class_list":["post-1821","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-new-models","tag-cactus-compute-en","tag-function-calling-en","tag-needle-3-en","tag-needle3-en","tag-tool-calling-en","tag-verified","tag--en"],"lang":"en","translations":{"en":1821,"ja":1819},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1821","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=1821"}],"version-history":[{"count":1,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1821\/revisions"}],"predecessor-version":[{"id":1840,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/1821\/revisions\/1840"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/1843"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=1821"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=1821"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=1821"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}