{"id":8555,"date":"2026-10-01T13:10:46","date_gmt":"2026-10-01T04:10:46","guid":{"rendered":"https:\/\/localmodelwatch.tsuchitsuchi.com\/2026\/10\/01\/magnitude-self-optimizing-inference-engine-for-agents\/"},"modified":"2026-10-06T07:16:00","modified_gmt":"2026-10-05T22:16:00","slug":"magnitude-self-optimizing-inference-engine-for-agents","status":"publish","type":"post","link":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/01\/magnitude-self-optimizing-inference-engine-for-agents\/","title":{"rendered":"Magnitude: Up to 2x Faster Than llama.cpp Officially, 21x Slower on Our CPU"},"content":{"rendered":"<p><!-- lmw:facts --><\/p>\n<h2>At a Glance<\/h2>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Value<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Published<\/td>\n<td>2026-10-01<\/td>\n<\/tr>\n<tr>\n<td>License<\/td>\n<td>Apache-2.0<\/td>\n<\/tr>\n<tr>\n<td>Source type<\/td>\n<td>Primary source (the publisher itself)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p><em>Values determined by this site&#8217;s code when the information was collected. Dates are JST.<\/em><\/p>\n<p><!-- \/lmw:facts --><\/p>\n<p><!-- lmw:engine-compare-summary --><\/p>\n<p><strong>What we checked ourselves<\/strong><\/p>\n<ul>\n<li>On our CPU-only server (Neoverse-N1, 3 cores) with the same model file (<code>Qwen3.5-4B-UD-Q4_K_XL.gguf<\/code>), Magnitude 0.2.3 generated 0.24\u20130.41 tokens\/s against 4.92\u20136.19 tokens\/s for llama.cpp b11223: about 21\u00d7 slower (median over our five questions).<\/li>\n<li><strong>We do not recommend Magnitude for CPU-only machines.<\/strong> The official \u201cup to 2\u00d7 faster than llama.cpp\u201d figures are for Metal and CUDA.<\/li>\n<\/ul>\n<p>The setup, conditions and the exact steps we ran are in \u201cOur Own Measurements\u201d below.<\/p>\n<p><!-- \/lmw:engine-compare-summary --><\/p>\n<h2>Overview<\/h2>\n<p>The initial release of Magnitude has been announced. Magnitude is an open-source inference engine (Apache-2.0 license) developed in Rust specifically for AI agents, featuring the ability to automatically optimize kernels according to the execution hardware.<\/p>\n<p>In this version, device-level kernel compilation and tuning achieve up to 2x faster inference speeds compared to llama.cpp, making it a very powerful choice for users looking to dramatically accelerate agent performance in local environments.<\/p>\n<h2>Key Changes<\/h2>\n<h3>Introduction of Automatic Device-Specific Kernel Optimization<\/h3>\n<p>Unlike typical inference engines that distribute pre-compiled kernels for a broad range of hardware, Magnitude compiles and tunes kernels at runtime to match the user&#8217;s actual device.<\/p>\n<p>This process generates code tailored perfectly to the characteristics of a specific chip. Whether using Apple Silicon, NVIDIA GPUs, AMD GPUs, or CPU-only environments, users can push their hardware&#8217;s native performance to the limit. Because tuning occurs before running a model, an optimization process runs upon initial startup, but subsequent inference speeds improve dramatically. This provides a particularly massive benefit to users with custom hardware configurations where existing general-purpose engines failed to unlock full performance.<\/p>\n<h3>Achieving Inference Performance That Outperforms llama.cpp<\/h3>\n<p>In benchmarks, it has recorded scores significantly higher than existing engines like llama.cpp. Specifically, a 92% improvement in decoding speed has been confirmed in Metal (Apple Silicon) environments, along with a 19% speedup in CUDA (NVIDIA) environments.<\/p>\n<p>This performance boost directly improves the user experience, especially in agent tasks where sequential token generation is critical. Because open-weight models can be run at speeds unattainable by general-purpose engines, more complex reasoning can be completed in less time. After installation, this high-speed inference environment can be utilized simply by selecting a recommended model from the &#8220;Discover&#8221; section within the app.<\/p>\n<h3>Memory Optimization and Cache Sharing for Parallel Agent Execution<\/h3>\n<p>Advanced memory management features have been implemented assuming workloads where multiple agents run simultaneously. Memory usage per agent has been reduced by 27% compared to conventional levels, and memory is released immediately once an agent finishes its task.<\/p>\n<p>Additionally, to enable fast parallel sessions, it features a function to share the prefix cache between sessions. This suppresses duplicate memory consumption and prevents processing stalls even when launching multiple agents using the same system prompt or background knowledge. For developers running multiple agents locally at the same time, this significantly reduces the burden of resource management.<\/p>\n<h3>Seamless Integration with Major Agents<\/h3>\n<p>One-click connection features have been introduced for widely used open-weight model-based agents such as Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.<\/p>\n<p>Users already utilizing these agents can benefit from a high-speed inference environment without configuration hassles simply by connecting through Magnitude&#8217;s &#8220;Connections&#8221; settings. Furthermore, because it provides an OpenAI-compatible API, it can also be easily used from other custom agents and tools through a standard interface.<\/p>\n<h3>Providing a Fully Local and Private Execution Environment<\/h3>\n<p>Magnitude is designed with privacy in mind, keeping all prompts, files, and models within the user&#8217;s local machine. No internet connection is required except when downloading models.<\/p>\n<p>Not only are token fees avoided, but there is no concern about data being sent to external servers even when operating agents handling sensitive data, making it an optimal solution for engineers looking to build a secure development environment. No special external service registration is required to get started; simply installing the desktop app makes all features, including the CLI, available.<\/p>\n<h2>Specifications<\/h2>\n<p>Magnitude provides an optimized execution environment for a wide variety of hardware setups and major open-weight models.<\/p>\n<p>On the hardware front, there are no fixed minimum spec requirements for operation. It supports diverse graphic accelerators including Apple Silicon (Mac), NVIDIA GPUs, and AMD GPUs, and can run without issues in CPU-only environments lacking a GPU. Because it operates flexibly according to the machine resources of the execution environment, users can carry out optimal operations suited to each environment, such as running smaller models on low-memory compact machines and larger models on advanced machines equipped with large memory capacities.<\/p>\n<p>Regarding model support, rather than a one-size-fits-all approach covering every architecture, it focuses on open-weight model families that have gained particular popularity in the developer community. By hand-writing dedicated, manually optimized kernels specifically for these models, it achieves inference speeds that vastly surpass general-purpose inference engines. The complete list of specifically supported models is progressively checked and updated on the official website ( <a href=\"https:\/\/magnitude.dev\/models\">magnitude.dev\/models<\/a> ).<\/p>\n<p>Furthermore, to accommodate AI agent use cases, connectivity with a broad ecosystem is secured. Supported agents that can be easily linked with a single click of the operation include Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Even for third-party agents or custom-developed tools not included in the list, communication can be conducted via standard OpenAI-compatible APIs to similarly leverage Magnitude&#8217;s high-speed inference engine.<\/p>\n<h2>How to Get It<\/h2>\n<p>Magnitude&#8217;s setup and onboarding steps are organized to be extremely simple, provided as an integrated application supporting major desktop OSs.<\/p>\n<p>Specific steps from installation to starting usage are as follows:<\/p>\n<ol>\n<li>\n<p><strong>Download and Launch the Desktop App<\/strong><br \/>\n   Download Magnitude provided for macOS, Windows, and Linux operating systems, install it on your environment, and launch the application. Because this desktop app bundles the <code>magnitude<\/code> CLI from the start, there is zero hassle of individually installing and configuring extra CLI tools for terminal operations.<\/p>\n<\/li>\n<li>\n<p><strong>Select and Download a Model<\/strong><br \/>\n   Open the &#8220;Discover&#8221; section in the app navigation. Here, select an open-weight model matching your hardware environment and use case from the recommended models, and download it to your local environment.<\/p>\n<\/li>\n<li>\n<p><strong>Connect Agents and Start Execution<\/strong><br \/>\n   Go to the &#8220;Connections&#8221; section in the app, select the agent you normally use (Pi, OpenCode, Hermes, Codex, etc.), and establish a connection. Because linking setup completes with a single click, you can immediately start inference processing through the agent.<\/p>\n<\/li>\n<\/ol>\n<p>Once model downloading is complete, all subsequent prompt processing, file referencing, and model execution happen entirely within the local machine. Because zero data transmission to external networks occurs, you can immediately start setup and operations in a private and high-speed environment.<\/p>\n<p><!-- lmw:engine-compare --><\/p>\n<h2>Our Own Measurements<\/h2>\n<h3>Magnitude vs llama.cpp on a CPU Only<\/h3>\n<p>Magnitude publishes \u201cup to 2\u00d7 faster than llama.cpp\u201d, but those figures are for Metal (Apple Silicon) and CUDA (NVIDIA); no CPU-only figures are published. We ran it on our server, which has no GPU, gave llama.cpp the same model file, and sent both the same questions in the same way.<\/p>\n<p><strong>We do not recommend Magnitude for CPU-only machines.<\/strong> Under these conditions its generation was about 21\u00d7 slower than llama.cpp (15\u201322\u00d7 per question), and prompt processing 16\u201338\u00d7 slower. If you run models on a CPU, llama.cpp (or a tool built on it) was by far the faster choice here.<\/p>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Item<\/th>\n<th>Condition<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>CPU<\/td>\n<td>Neoverse-N1 (no GPU), 3 cores<\/td>\n<\/tr>\n<tr>\n<td>Model<\/td>\n<td><code>Qwen3.5-4B-UD-Q4_K_XL.gguf<\/code> from <code>unsloth\/Qwen3.5-4B-MTP-GGUF<\/code> (revision <code>86835bf9<\/code>), the file Magnitude&#8217;s catalog uses for <code>qwen3.5-4b:gguf:q4<\/code>; both engines read the same file<\/td>\n<\/tr>\n<tr>\n<td>Magnitude<\/td>\n<td>0.2.3 (<code>magnitude-inference<\/code>, native build <code>0.1.0+src.3cc3838e83767a1f232f223b<\/code>). Context fixed at 262,144 tokens; MTP speculative decoding on (automatic)<\/td>\n<\/tr>\n<tr>\n<td>llama.cpp<\/td>\n<td>b11223 (official ubuntu-arm64 build), context 4,096 tokens; with and without MTP speculative decoding (<code>--spec-type draft-mtp<\/code>)<\/td>\n<\/tr>\n<tr>\n<td>Questions<\/td>\n<td>Our five fixed Japanese questions (version 2), temperature 0, up to 512 tokens, thinking off, streaming<\/td>\n<\/tr>\n<tr>\n<td>Date<\/td>\n<td>2026-10-01<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<div class=\"lmw-table-scroll\" tabindex=\"0\" style=\"overflow-x:auto;-webkit-overflow-scrolling:touch;max-width:100%;\">\n<p class=\"lmw-table-hint\" style=\"margin:0 0 4px;font-size:0.85em;opacity:0.7;\">\u2192 Scroll horizontally to see all columns<\/p>\n<table style=\"width:max-content;min-width:100%;border-collapse:collapse;\">\n<thead>\n<tr>\n<th>Question<\/th>\n<th>Magnitude generation<\/th>\n<th>llama.cpp generation<\/th>\n<th>llama.cpp + MTP generation<\/th>\n<th>Magnitude: time per answer<\/th>\n<th>llama.cpp: time per answer<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Summarization<\/td>\n<td>0.24 tok\/s<\/td>\n<td>4.92 tok\/s<\/td>\n<td>4.83 tok\/s<\/td>\n<td>33 min<\/td>\n<td>67 s<\/td>\n<\/tr>\n<tr>\n<td>Formal business email<\/td>\n<td>0.25 tok\/s<\/td>\n<td>5.68 tok\/s<\/td>\n<td>5.04 tok\/s<\/td>\n<td>28 min<\/td>\n<td>93 s<\/td>\n<\/tr>\n<tr>\n<td>Word problem<\/td>\n<td>0.41 tok\/s<\/td>\n<td>5.99 tok\/s<\/td>\n<td>6.64 tok\/s<\/td>\n<td>18 min<\/td>\n<td>89 s<\/td>\n<\/tr>\n<tr>\n<td>Knowledge<\/td>\n<td>0.31 tok\/s<\/td>\n<td>5.72 tok\/s<\/td>\n<td>6.27 tok\/s<\/td>\n<td>9 min<\/td>\n<td>92 s<\/td>\n<\/tr>\n<tr>\n<td>Constrained writing<\/td>\n<td>0.29 tok\/s<\/td>\n<td>6.19 tok\/s<\/td>\n<td>5.08 tok\/s<\/td>\n<td>11 min<\/td>\n<td>38 s<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Other things we observed:<\/p>\n<ul>\n<li>Prompt processing: Magnitude 0.49\u20130.79 tok\/s, llama.cpp 12.91\u201318.65 tok\/s.<\/li>\n<li>Preparing to load: every load first compiled and tuned kernels for this CPU, which took 33\u201348 minutes across our 3 loads.<\/li>\n<li>Memory: the context is fixed at the model&#8217;s maximum (262,144 tokens) and the load plan asked for 9.9GB. Loads with a 12 GB and 14 GB memory limit were refused for lack of memory after the preparation finished; with 16GB it loaded (peak 12.2GB).<\/li>\n<li>Streaming: even with streaming requested, Magnitude&#8217;s text arrived all at once at the end of each answer (first text after 9\u201333 minutes); llama.cpp&#8217;s first text came after 2\u201339 seconds.<\/li>\n<li>Magnitude&#8217;s MTP accepted 12%\u201345% of the drafted tokens.<\/li>\n<\/ul>\n<p>Please note:<\/p>\n<ul>\n<li>These are figures from one CPU-only ARM server. They are not a guide to speed on a Mac (Metal) or an NVIDIA GPU, and they do not contradict the official figures for those.<\/li>\n<li>Magnitude is normally used through its desktop app or <code>magnitude serve<\/code>; we started the component that does the inference in those, <code>magnitude-inference<\/code>, on its own.<\/li>\n<li>We ran Magnitude (a prebuilt binary) isolated from the rest of our server (a dedicated user, no network, 3 CPU cores) and ran llama.cpp in the same sandbox.<\/li>\n<li>We compared speed only, not the content of the answers.<\/li>\n<\/ul>\n<h4>Setup, Download and Run Steps (What We Actually Ran)<\/h4>\n<p>Run these in an empty working folder. <code>icn\/installation.json<\/code> is a file we wrote ourselves (its format comes from <code>inference\/service\/server\/src\/installation.rs<\/code> in the source). We ran each <code>python3 bench.py<\/code> inside the sandbox described above (the settings are listed at the end).<\/p>\n<pre><code class=\"language-bash\"># 1. Download Magnitude 0.2.3's inference component (Linux ARM64), check its SHA-256 against the release record\ncurl -LO 'https:\/\/github.com\/magnitudedev\/magnitude\/releases\/download\/%40magnitudedev%2Fcli%400.2.3\/magnitude-release.json'\ncurl -LO 'https:\/\/github.com\/magnitudedev\/magnitude\/releases\/download\/%40magnitudedev%2Fcli%400.2.3\/magnitude-icn-base-linux-arm64-gnu.tar.gz'\necho 'b0824a230b1b90542a3a4d2d95d278d5f655ef3305b918b88964e1e48ed34c1a  magnitude-icn-base-linux-arm64-gnu.tar.gz' | sha256sum -c -\nmkdir -p icn &amp;&amp; tar xzf magnitude-icn-base-linux-arm64-gnu.tar.gz -C icn\necho '{&quot;schemaVersion&quot;:1,&quot;nativeBuild&quot;:&quot;0.1.0+src.3cc3838e83767a1f232f223b&quot;}' &gt; icn\/installation.json\nmkdir -p store cache out\n\n# 2. llama.cpp b11223 (the official ubuntu-arm64 build)\ncurl -LO 'https:\/\/github.com\/ggml-org\/llama.cpp\/releases\/download\/b11223\/llama-b11223-bin-ubuntu-arm64.tar.gz'\necho 'e4280e5c71369a04bef5a171e50ab77fd923f6c86318106b19942397c4ded75f  llama-b11223-bin-ubuntu-arm64.tar.gz' | sha256sum -c -\ntar xzf llama-b11223-bin-ubuntu-arm64.tar.gz &amp;&amp; mv llama-b11223 llama\n\n# 3. Place the model (the revision pinned by Magnitude's catalog) in Hugging Face cache layout\nREV=86835bf9949e4d14d6860f7910b1340ad4f271a9\nH=hf\/hub\/models--unsloth--Qwen3.5-4B-MTP-GGUF\nmkdir -p $H\/blobs $H\/snapshots\/$REV $H\/refs &amp;&amp; echo -n $REV &gt; $H\/refs\/main\ncurl -s &quot;https:\/\/huggingface.co\/api\/models\/unsloth\/Qwen3.5-4B-MTP-GGUF\/revision\/$REV?blobs=true&quot; | python3 -c &quot;\nimport json,sys;d=json.load(sys.stdin)\nfor s in d['siblings']:\n  if s['rfilename'] in ('Qwen3.5-4B-UD-Q4_K_XL.gguf','mmproj-BF16.gguf'): print(s['rfilename'], s['lfs']['sha256'], s['size'])&quot; &gt; files.txt\nwhile read name sha size; do\n  curl -sfL -o $H\/blobs\/$sha &quot;https:\/\/huggingface.co\/unsloth\/Qwen3.5-4B-MTP-GGUF\/resolve\/$REV\/$name&quot;\n  echo &quot;$sha  $H\/blobs\/$sha&quot; | sha256sum -c -\n  ln -sfn ..\/..\/blobs\/$sha $H\/snapshots\/$REV\/$name\ndone &lt; files.txt\n\n# 4. Send the same questions to the three setups (bench.py below)\npython3 bench.py magnitude 2 512\npython3 bench.py llama 1 512\npython3 bench.py llama-mtp 1 512\n<\/code><\/pre>\n<p><code>bench.py<\/code> (written by us, in full) starts each engine, sends the five questions with streaming, and records the speeds each engine reports. Files go under the working folder:<\/p>\n<pre><code class=\"language-python\">&quot;&quot;&quot;Magnitude \u3068 llama-server \u306b\u540c\u3058\u8cea\u554f\u3092\u540c\u3058\u65b9\u6cd5\u3067\u9001\u308a\u3001\u901f\u3055\u3092\u6bd4\u3079\u308b(\u7bb1\u306e\u4e2d\u3067\u52d5\u304b\u3059)\u3002&quot;&quot;&quot;\nimport json, subprocess, sys, time, urllib.request, os\nM = os.path.abspath(&quot;.&quot;)\nGGUF = f&quot;{M}\/hf\/hub\/models--unsloth--Qwen3.5-4B-MTP-GGUF\/snapshots\/86835bf9949e4d14d6860f7910b1340ad4f271a9\/Qwen3.5-4B-UD-Q4_K_XL.gguf&quot;\nengine = sys.argv[1]\nrounds = int(sys.argv[2]) if len(sys.argv) &gt; 2 else 2\nmax_tokens = int(sys.argv[3]) if len(sys.argv) &gt; 3 else 512\nPORT = 18080\nif engine == &quot;magnitude&quot;:\n    cmd = [f&quot;{M}\/icn\/bin\/magnitude-inference&quot;, &quot;serve&quot;, &quot;--bind&quot;, f&quot;127.0.0.1:{PORT}&quot;,\n           &quot;--model-store&quot;, f&quot;{M}\/store&quot;, &quot;--cache-root&quot;, f&quot;{M}\/cache&quot;,\n           &quot;--hf-cache&quot;, f&quot;{M}\/hf\/hub&quot;, &quot;--installation&quot;, f&quot;{M}\/icn\/installation.json&quot;]\n    env = dict(os.environ)\nelse:\n    b = f&quot;{M}\/llama&quot;\n    cmd = [f&quot;{b}\/llama-server&quot;, &quot;-m&quot;, GGUF, &quot;-t&quot;, &quot;3&quot;, &quot;-c&quot;, &quot;4096&quot;, &quot;--jinja&quot;, &quot;--no-mmproj&quot;,\n           &quot;--host&quot;, &quot;127.0.0.1&quot;, &quot;--port&quot;, str(PORT)]\n    if engine == &quot;llama-mtp&quot;:\n        cmd += [&quot;--spec-type&quot;, &quot;draft-mtp&quot;]\n    env = dict(os.environ, LD_LIBRARY_PATH=b)\nlog = open(f&quot;{M}\/out\/{engine}.log&quot;, &quot;w&quot;)\np = subprocess.Popen(cmd, stdout=log, stderr=subprocess.STDOUT, env=env)\n\ndef call(path, data=None, timeout=60):\n    req = urllib.request.Request(f&quot;http:\/\/127.0.0.1:{PORT}{path}&quot;,\n                                 data=json.dumps(data).encode() if data is not None else None,\n                                 headers={&quot;Content-Type&quot;: &quot;application\/json&quot;},\n                                 method=&quot;POST&quot; if data is not None else &quot;GET&quot;)\n    with urllib.request.urlopen(req, timeout=timeout) as r:\n        return json.loads(r.read() or b&quot;null&quot;)\n\ndef ask(model, text):\n    body = {&quot;model&quot;: model, &quot;messages&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: text}], &quot;temperature&quot;: 0, &quot;seed&quot;: 0,\n            &quot;max_tokens&quot;: max_tokens, &quot;stream&quot;: True, &quot;stream_options&quot;: {&quot;include_usage&quot;: True}}\n    if engine == &quot;magnitude&quot;:\n        body[&quot;reasoning_effort&quot;] = &quot;none&quot;\n    else:\n        body[&quot;chat_template_kwargs&quot;] = {&quot;enable_thinking&quot;: False}\n    req = urllib.request.Request(f&quot;http:\/\/127.0.0.1:{PORT}\/v1\/chat\/completions&quot;, data=json.dumps(body).encode(),\n                                 headers={&quot;Content-Type&quot;: &quot;application\/json&quot;}, method=&quot;POST&quot;)\n    t0 = time.time(); first = None; last = None; chunks = 0; usage = None; timings = None; text_out = []; finish = None\n    with urllib.request.urlopen(req, timeout=3600) as r:\n        for raw in r:\n            line = raw.decode(&quot;utf-8&quot;, &quot;replace&quot;).strip()\n            if not line.startswith(&quot;data:&quot;):\n                continue\n            payload = line[5:].strip()\n            if payload == &quot;[DONE]&quot;:\n                break\n            ev = json.loads(payload)\n            usage = ev.get(&quot;usage&quot;) or usage\n            timings = ev.get(&quot;timings&quot;) or timings\n            for ch in ev.get(&quot;choices&quot;) or []:\n                d = ch.get(&quot;delta&quot;) or {}\n                piece = (d.get(&quot;content&quot;) or &quot;&quot;) + (d.get(&quot;reasoning_content&quot;) or d.get(&quot;reasoning&quot;) or &quot;&quot;)\n                finish = ch.get(&quot;finish_reason&quot;) or finish\n                if piece:\n                    now = time.time()\n                    first = first or now\n                    last = now\n                    chunks += 1\n                    text_out.append(d.get(&quot;content&quot;) or &quot;&quot;)\n    n = (usage or {}).get(&quot;completion_tokens&quot;) or chunks\n    res = {&quot;ttft_s&quot;: round(first - t0, 3) if first else None,\n           &quot;completion_tokens&quot;: n, &quot;chunks&quot;: chunks, &quot;finish&quot;: finish,\n           &quot;prompt_tokens&quot;: (usage or {}).get(&quot;prompt_tokens&quot;),\n           &quot;decode_tps&quot;: round((n - 1) \/ (last - first), 2) if first and last and last &gt; first else None,\n           &quot;wall_s&quot;: round(time.time() - t0, 2), &quot;head&quot;: &quot;&quot;.join(text_out)[:80]}\n    if timings:\n        res[&quot;native&quot;] = {k: timings.get(k) for k in (&quot;prompt_n&quot;, &quot;prompt_per_second&quot;, &quot;predicted_n&quot;, &quot;predicted_per_second&quot;, &quot;draft_n&quot;, &quot;draft_n_accepted&quot;)}\n    return res\n\nout = {&quot;engine&quot;: engine, &quot;rounds&quot;: []}\ntry:\n    t_start = time.time()\n    for _ in range(600):\n        if p.poll() is not None:\n            raise SystemExit(&quot;server exited&quot;)\n        try:\n            if call(&quot;\/health&quot;, timeout=5): break\n        except Exception:\n            time.sleep(1)\n    if engine == &quot;magnitude&quot;:\n        model = &quot;qwen3.5-4b:gguf:q4&quot;\n        try: print(&quot;plan&quot;, json.dumps(call(f&quot;\/api\/v1\/models\/{model}\/load-plan&quot;, {}), ensure_ascii=False)[:1500], flush=True)\n        except Exception as e: print(&quot;plan ERR&quot;, e, getattr(e, &quot;read&quot;, lambda: b&quot;&quot;)()[:800], flush=True)\n        t = time.time()\n        inst = call(&quot;\/api\/v1\/instances&quot;, {&quot;modelId&quot;: model}, timeout=7200)\n        out[&quot;load_s&quot;] = round(time.time() - t, 1)\n        print(&quot;instance&quot;, json.dumps(inst, ensure_ascii=False)[:1500], flush=True)\n    else:\n        model = &quot;qwen&quot;\n    out[&quot;ready_s&quot;] = round(time.time() - t_start, 1)\n    prompts = json.load(open(f&quot;{M}\/prompts.json&quot;))[&quot;prompts&quot;]\n    for r in range(rounds):\n        rows = []\n        for pr in prompts:\n            res = ask(model, pr[&quot;ja&quot;]); res[&quot;id&quot;] = pr[&quot;id&quot;]; rows.append(res)\n            print(r, json.dumps(res, ensure_ascii=False), flush=True)\n        out[&quot;rounds&quot;].append(rows)\nfinally:\n    p.terminate()\n    try: p.wait(60)\n    except Exception: p.kill()\n    json.dump(out, open(f&quot;{M}\/out\/{engine}.json&quot;, &quot;w&quot;), ensure_ascii=False, indent=1)\n<\/code><\/pre>\n<p><code>prompts.json<\/code> placed in the working folder (our fixed questions, in full):<\/p>\n<pre><code class=\"language-json\">{\n  &quot;version&quot;: 2,\n  &quot;note&quot;: &quot;\u5f53\u30b5\u30a4\u30c8\u3067\u306e\u5b9f\u6e2c(lmw\/lab\/cpu_bench.py)\u3067\u5168\u30e2\u30c7\u30eb\u306b\u540c\u3058\u6761\u4ef6\u3067\u5c0b\u306d\u308b\u65e5\u672c\u8a9e\u306e\u8cea\u554f\u3002\u5909\u3048\u308b\u3068\u304d\u306f version \u3092\u4e0a\u3052\u308b(\u904e\u53bb\u306e\u56de\u7b54\u3068\u6bd4\u3079\u3089\u308c\u306a\u304f\u306a\u308b\u305f\u3081\u3001\u5b9f\u6e2c\u306f\u53d6\u308a\u76f4\u3057\u306b\u306a\u308b)\u3002reference \u306f\u7b54\u3048\u304c1\u3064\u306b\u6c7a\u307e\u308b\u8cea\u554f\u306e\u6b63\u89e3\u3067\u3001\u4eba\u9593\u304c\u78ba\u304b\u3081\u305f\u4e8b\u5b9f\u3060\u3051\u3092\u66f8\u304f\u3002\u63a1\u70b9\u306b\u306f\u4f7f\u308f\u305a\u3001\u8aad\u8005\u304c\u898b\u6bd4\u3079\u308b\u305f\u3081\u306b\u8868\u793a\u3059\u308b\u3002 2026-09-28 v2: \u8981\u7d04\u306e\u5143\u306e\u6587\u7ae0\u3092\u77ed\u3044\u6bb5\u843d(\u7d04200\u5b57)\u304b\u3089\u66f8\u304d\u4e0b\u308d\u3057\u306e\u8a18\u4e8b(\u7d041,200\u5b57)\u306b\u66ff\u3048\u305f(\u30e6\u30fc\u30b6\u30fc\u306e\u6307\u6458: \u77ed\u3059\u304e\u3066\u8981\u7d04\u3067\u304d\u305f\u304b\u5224\u65ad\u3067\u304d\u306a\u3044)\u3002&quot;,\n  &quot;prompts&quot;: [\n    {\n      &quot;id&quot;: &quot;summary&quot;,\n      &quot;label&quot;: {\n        &quot;ja&quot;: &quot;\u8981\u7d04&quot;,\n        &quot;en&quot;: &quot;Summarization&quot;\n      },\n      &quot;ja&quot;: &quot;\u6b21\u306e\u8a18\u4e8b\u30923\u6587\u3067\u8981\u7d04\u3057\u3066\u304f\u3060\u3055\u3044\u3002\\n\\n\u307f\u3069\u308a\u5e02\u306f\u4eca\u5e74\u306e\u6625\u3001\u5e02\u5185\u306b\u4e09\u3064\u3042\u308b\u516c\u7acb\u56f3\u66f8\u9928\u306e\u3046\u3061\u6700\u3082\u53e4\u3044\u4e2d\u592e\u56f3\u66f8\u9928\u306e\u6539\u4fee\u3092\u7d42\u3048\u3001\u5341\u6708\u4e00\u65e5\u306b\u518d\u958b\u9928\u3057\u305f\u3002\u6539\u4fee\u306f\u7d04\u4e00\u5e74\u534a\u306b\u308f\u305f\u308a\u3001\u7dcf\u4e8b\u696d\u8cbb\u306f\u7d04\u5341\u4e8c\u5104\u5186\u3002\u3053\u306e\u3046\u3061\u7d04\u56db\u5104\u5186\u306f\u56fd\u306e\u88dc\u52a9\u91d1\u3067\u307e\u304b\u306a\u308f\u308c\u305f\u3002\\n\\n\u6700\u5927\u306e\u5909\u5316\u306f\u3001\u95b2\u89a7\u5e2d\u306e\u914d\u7f6e\u3067\u3042\u308b\u3002\u4ee5\u524d\u306f\u9928\u5185\u306e\u5ea7\u5e2d\u306e\u5927\u534a\u304c\u5b66\u7fd2\u7528\u306e\u673a\u3067\u3001\u8a66\u9a13\u671f\u9593\u306b\u306a\u308b\u3068\u4e2d\u9ad8\u751f\u3067\u57cb\u307e\u308a\u3001\u9ad8\u9f62\u306e\u5229\u7528\u8005\u3084\u5c0f\u3055\u306a\u5b50\u3069\u3082\u9023\u308c\u304c\u5ea7\u308c\u306a\u3044\u3068\u3044\u3046\u82e6\u60c5\u304c\u6bce\u5e74\u5bc4\u305b\u3089\u308c\u3066\u3044\u305f\u3002\u6539\u4fee\u5f8c\u306f\u5ea7\u5e2d\u6570\u3092\u767e\u4e8c\u5341\u5e2d\u304b\u3089\u4e8c\u767e\u5e2d\u306b\u5897\u3084\u3057\u305f\u3046\u3048\u3067\u3001\u5b66\u7fd2\u7528\u306e\u673a\u3092\u4e09\u968e\u306b\u307e\u3068\u3081\u3001\u4e00\u968e\u306f\u4f1a\u8a71\u304c\u3067\u304d\u308b\u300c\u306b\u304e\u308f\u3044\u30d5\u30ed\u30a2\u300d\u3001\u4e8c\u968e\u306f\u9759\u304b\u306b\u672c\u3092\u8aad\u3080\u300c\u95b2\u89a7\u30d5\u30ed\u30a2\u300d\u306b\u5206\u3051\u305f\u3002\u4e00\u968e\u306b\u306f\u4e88\u7d04\u5236\u306e\u30b0\u30eb\u30fc\u30d7\u5b66\u7fd2\u5ba4\u304c\u56db\u5ba4\u3068\u3001\u98f2\u307f\u7269\u3092\u6301\u3061\u8fbc\u3081\u308b\u30ab\u30d5\u30a7\u30b9\u30da\u30fc\u30b9\u3082\u8a2d\u3051\u3089\u308c\u305f\u3002\\n\\n\u8535\u66f8\u306e\u9762\u3067\u3082\u898b\u76f4\u3057\u304c\u9032\u3093\u3060\u3002\u8cb8\u51fa\u306e\u5c11\u306a\u3044\u53e4\u3044\u8cc7\u6599\u7d04\u4e09\u4e07\u518a\u3092\u5e02\u5185\u306e\u4ed6\u306e\u9928\u3084\u4fdd\u5b58\u66f8\u5eab\u306b\u79fb\u3057\u3001\u4ee3\u308f\u308a\u306b\u5b50\u3069\u3082\u5411\u3051\u306e\u7d75\u672c\u3068\u3001\u4ecb\u8b77\u30fb\u5b50\u80b2\u3066\u30fb\u4ed5\u4e8b\u63a2\u3057\u3068\u3044\u3063\u305f\u66ae\u3089\u3057\u306b\u95a2\u308f\u308b\u5b9f\u7528\u66f8\u3092\u7d04\u4e00\u4e07\u4e94\u5343\u518a\u5897\u3084\u3057\u305f\u3002\u96fb\u5b50\u66f8\u7c4d\u306e\u8cb8\u51fa\u30b5\u30fc\u30d3\u30b9\u3082\u59cb\u307e\u308a\u3001\u5229\u7528\u767b\u9332\u3092\u3059\u308c\u3070\u81ea\u5b85\u304b\u3089\u4e00\u5ea6\u306b\u4e09\u518a\u307e\u3067\u3001\u4e8c\u9031\u9593\u501f\u308a\u3089\u308c\u308b\u3002\\n\\n\u5efa\u7269\u305d\u306e\u3082\u306e\u306b\u3082\u624b\u304c\u5165\u3063\u305f\u3002\u7bc9\u56db\u5341\u5e74\u3092\u8d85\u3048\u3066\u3044\u305f\u7a7a\u8abf\u8a2d\u5099\u3092\u5165\u308c\u66ff\u3048\u3001\u5c4b\u4e0a\u306b\u306f\u592a\u967d\u5149\u30d1\u30cd\u30eb\u3092\u8a2d\u7f6e\u3057\u3066\u3001\u9928\u5185\u3067\u4f7f\u3046\u96fb\u529b\u306e\u7d04\u4e8c\u5272\u3092\u307e\u304b\u306a\u3048\u308b\u3088\u3046\u306b\u3057\u305f\u3002\u8eca\u3044\u3059\u306e\u5229\u7528\u8005\u304b\u3089\u8981\u671b\u306e\u591a\u304b\u3063\u305f\u30a8\u30ec\u30d9\u30fc\u30bf\u30fc\u306f\u4e00\u57fa\u304b\u3089\u4e8c\u57fa\u306b\u5897\u3048\u3001\u66f8\u67b6\u306e\u9593\u306e\u901a\u8def\u3082\u4ee5\u524d\u3088\u308a\u4e09\u5341\u30bb\u30f3\u30c1\u5e83\u3052\u3089\u308c\u305f\u3002\u5165\u53e3\u306b\u306f\u6bb5\u5dee\u306e\u306a\u3044\u30b9\u30ed\u30fc\u30d7\u304c\u65b0\u305f\u306b\u8a2d\u3051\u3089\u308c\u3001\u6388\u4e73\u5ba4\u3068\u591a\u76ee\u7684\u30c8\u30a4\u30ec\u3082\u5404\u968e\u306b\u4e00\u3064\u305a\u3064\u7f6e\u304b\u308c\u305f\u3002\\n\\n\u4e00\u65b9\u3067\u8ab2\u984c\u3082\u6b8b\u308b\u3002\u99d0\u8eca\u5834\u306f\u6539\u4fee\u524d\u3068\u540c\u3058\u56db\u5341\u53f0\u306e\u307e\u307e\u3067\u3001\u9031\u672b\u306b\u306f\u8fd1\u304f\u306e\u9053\u8def\u306b\u9806\u756a\u5f85\u3061\u306e\u8eca\u304c\u4e26\u3093\u3060\u3002\u5e02\u306f\u96a3\u63a5\u3059\u308b\u5e02\u6709\u5730\u3092\u81e8\u6642\u99d0\u8eca\u5834\u3068\u3057\u3066\u958b\u653e\u3057\u305f\u304c\u3001\u6052\u4e45\u7684\u306a\u5bfe\u7b56\u306f\u6c7a\u307e\u3063\u3066\u3044\u306a\u3044\u3002\u307e\u305f\u3001\u958b\u9928\u6642\u9593\u3092\u5e73\u65e5\u306e\u591c\u4e5d\u6642\u307e\u3067\u5ef6\u3070\u3057\u305f\u3053\u3068\u3067\u8077\u54e1\u306e\u8ca0\u62c5\u304c\u5897\u3048\u3066\u304a\u308a\u3001\u8077\u54e1\u56e3\u4f53\u306f\u4eba\u54e1\u306e\u8ffd\u52a0\u3092\u6c42\u3081\u3066\u3044\u308b\u3002\\n\\n\u5229\u7528\u8005\u306e\u53d7\u3051\u6b62\u3081\u306f\u304a\u304a\u3080\u306d\u597d\u610f\u7684\u3060\u3002\u9031\u306b\u4e8c\u5ea6\u3001\u5b6b\u3092\u9023\u308c\u3066\u6765\u308b\u3068\u3044\u3046\u4e03\u5341\u4ee3\u306e\u5973\u6027\u306f\u300c\u524d\u306f\u5b50\u3069\u3082\u304c\u58f0\u3092\u51fa\u3059\u3068\u5468\u308a\u306e\u76ee\u304c\u6c17\u306b\u306a\u3063\u305f\u304c\u3001\u4e00\u968e\u306a\u3089\u6c17\u517c\u306d\u306a\u304f\u7d75\u672c\u3092\u8aad\u3093\u3067\u3042\u3052\u3089\u308c\u308b\u300d\u3068\u8a71\u3059\u3002\u4e00\u65b9\u3001\u53d7\u9a13\u3092\u63a7\u3048\u305f\u9ad8\u6821\u751f\u306e\u4e2d\u306b\u306f\u300c\u5b66\u7fd2\u5e2d\u304c\u4e09\u968e\u3060\u3051\u306b\u306a\u308a\u3001\u5915\u65b9\u306f\u3059\u3050\u306b\u6e80\u5e2d\u306b\u306a\u308b\u300d\u3068\u4e0d\u6e80\u3092\u6f0f\u3089\u3059\u4eba\u3082\u3044\u308b\u3002\u56f3\u66f8\u9928\u306f\u3001\u5b66\u7fd2\u5e2d\u306e\u7a7a\u304d\u72b6\u6cc1\u3092\u30db\u30fc\u30e0\u30da\u30fc\u30b8\u3067\u78ba\u8a8d\u3067\u304d\u308b\u4ed5\u7d44\u307f\u3092\u5e74\u5185\u306b\u5c0e\u5165\u3059\u308b\u4e88\u5b9a\u3060\u3002\\n\\n\u518d\u958b\u9928\u306b\u5408\u308f\u305b\u3066\u3001\u50ac\u3057\u3082\u5897\u3048\u305f\u3002\u6bce\u9031\u571f\u66dc\u306e\u5348\u524d\u306b\u306f\u8aad\u307f\u805e\u304b\u305b\u4f1a\u3001\u6708\u306b\u4e00\u5ea6\u306f\u5730\u5143\u306e\u8fb2\u5bb6\u3084\u5546\u5e97\u4e3b\u3092\u8b1b\u5e2b\u306b\u62db\u304f\u300c\u307e\u3061\u306e\u5148\u751f\u300d\u8b1b\u5ea7\u304c\u958b\u304b\u308c\u308b\u3002\u5341\u4e00\u6708\u306b\u306f\u3001\u5e02\u5185\u306e\u4e2d\u5b66\u751f\u304c\u9078\u3093\u3060\u672c\u3092\u7d39\u4ecb\u3059\u308b\u5c55\u793a\u3082\u4e88\u5b9a\u3055\u308c\u3066\u3044\u308b\u3002\\n\\n\u9928\u9577\u306e\u5c71\u672c\u548c\u5b50\u3055\u3093\u306f\u300c\u672c\u3092\u501f\u308a\u308b\u5834\u6240\u304b\u3089\u3001\u4eba\u304c\u96c6\u307e\u3063\u3066\u904e\u3054\u3059\u5834\u6240\u3078\u5909\u3048\u305f\u304b\u3063\u305f\u3002\u518d\u958b\u9928\u304b\u3089\u306e\u4e00\u304b\u6708\u3067\u6765\u9928\u8005\u306f\u4ee5\u524d\u306e\u4e00\u30fb\u516b\u500d\u306b\u306a\u3063\u305f\u300d\u3068\u8a71\u3059\u3002\u5e02\u306f\u6765\u5e74\u5ea6\u3001\u6b8b\u308b\u4e8c\u3064\u306e\u56f3\u66f8\u9928\u306b\u3064\u3044\u3066\u3082\u5229\u7528\u8005\u3078\u306e\u805e\u304d\u53d6\u308a\u3092\u884c\u3044\u3001\u6539\u4fee\u306e\u5fc5\u8981\u6027\u3092\u691c\u8a0e\u3059\u308b\u65b9\u91dd\u3060\u3002&quot;,\n      &quot;en&quot;: &quot;Summarize the following article in three sentences. (The article, about 1,200 Japanese characters, describes the renovation and reopening of a city's central library: the budget, the new floor layout and seating, changes to the collection and e-book lending, building and accessibility work, remaining issues such as parking and staff workload, users' reactions, new events, and the city's next steps.)&quot;\n    },\n    {\n      &quot;id&quot;: &quot;keigo&quot;,\n      &quot;label&quot;: {\n        &quot;ja&quot;: &quot;\u656c\u8a9e\u306e\u30e1\u30fc\u30eb&quot;,\n        &quot;en&quot;: &quot;Formal business email&quot;\n      },\n      &quot;ja&quot;: &quot;\u53d6\u5f15\u5148\u306e\u62c5\u5f53\u8005\u306b\u3001\u6765\u9031\u706b\u66dc\u65e5\u306e\u6253\u3061\u5408\u308f\u305b\u3092\u6c34\u66dc\u65e5\u306b\u5909\u66f4\u3057\u3066\u307b\u3057\u3044\u3068\u304a\u9858\u3044\u3059\u308b\u30e1\u30fc\u30eb\u3092\u3001\u4e01\u5be7\u306a\u656c\u8a9e\u3067\u66f8\u3044\u3066\u304f\u3060\u3055\u3044\u3002&quot;,\n      &quot;en&quot;: &quot;Write an email in polite Japanese (keigo) asking a client's contact person to move next Tuesday's meeting to Wednesday.&quot;\n    },\n    {\n      &quot;id&quot;: &quot;arithmetic&quot;,\n      &quot;label&quot;: {\n        &quot;ja&quot;: &quot;\u6587\u7ae0\u984c&quot;,\n        &quot;en&quot;: &quot;Word problem&quot;\n      },\n      &quot;ja&quot;: &quot;\u308a\u3093\u3054\u304c1\u500b120\u5186\u3001\u307f\u304b\u3093\u304c1\u500b45\u5186\u3067\u3059\u3002\u308a\u3093\u3054\u30923\u500b\u3068\u307f\u304b\u3093\u30928\u500b\u8cb7\u3063\u3066\u30011,000\u5186\u672d\u3067\u6255\u3044\u307e\u3057\u305f\u3002\u304a\u3064\u308a\u306f\u3044\u304f\u3089\u3067\u3059\u304b\u3002\u8003\u3048\u65b9\u3082\u793a\u3057\u3066\u304f\u3060\u3055\u3044\u3002&quot;,\n      &quot;en&quot;: &quot;Apples cost 120 yen each and mandarins cost 45 yen each. You buy 3 apples and 8 mandarins and pay with a 1,000-yen bill. How much change do you get? Show your reasoning.&quot;,\n      &quot;reference&quot;: {\n        &quot;ja&quot;: &quot;\u6b63\u89e3\u306f280\u5186\u3067\u3059(120\u00d73=360\u5186\u300145\u00d78=360\u5186\u30011,000\u2212720=280\u5186)\u3002&quot;,\n        &quot;en&quot;: &quot;The correct answer is 280 yen (120\u00d73 = 360, 45\u00d78 = 360, 1,000 \u2212 720 = 280).&quot;\n      }\n    },\n    {\n      &quot;id&quot;: &quot;prefectures&quot;,\n      &quot;label&quot;: {\n        &quot;ja&quot;: &quot;\u77e5\u8b58&quot;,\n        &quot;en&quot;: &quot;Knowledge&quot;\n      },\n      &quot;ja&quot;: &quot;\u65e5\u672c\u306e\u90fd\u9053\u5e9c\u770c\u306e\u3046\u3061\u3001\u6d77\u306b\u9762\u3057\u3066\u3044\u306a\u3044\u770c\u3092\u3059\u3079\u3066\u6319\u3052\u3066\u304f\u3060\u3055\u3044\u3002&quot;,\n      &quot;en&quot;: &quot;List all of Japan's prefectures that do not border the sea.&quot;,\n      &quot;reference&quot;: {\n        &quot;ja&quot;: &quot;\u6b63\u89e3\u306f8\u770c\u3067\u3059(\u6803\u6728\u770c\u30fb\u7fa4\u99ac\u770c\u30fb\u57fc\u7389\u770c\u30fb\u5c71\u68a8\u770c\u30fb\u9577\u91ce\u770c\u30fb\u5c90\u961c\u770c\u30fb\u6ecb\u8cc0\u770c\u30fb\u5948\u826f\u770c)\u3002&quot;,\n        &quot;en&quot;: &quot;The correct answer is eight prefectures: Tochigi, Gunma, Saitama, Yamanashi, Nagano, Gifu, Shiga and Nara.&quot;\n      }\n    },\n    {\n      &quot;id&quot;: &quot;haiku&quot;,\n      &quot;label&quot;: {\n        &quot;ja&quot;: &quot;\u6307\u793a\u3069\u304a\u308a\u306e\u5275\u4f5c&quot;,\n        &quot;en&quot;: &quot;Constrained writing&quot;\n      },\n      &quot;ja&quot;: &quot;\u300c\u6625\u300d\u3092\u30c6\u30fc\u30de\u306b\u3057\u305f\u4ff3\u53e5\u30923\u3064\u3001\u4e94\u30fb\u4e03\u30fb\u4e94\u306e\u97f3\u6570\u3092\u5b88\u3063\u3066\u4f5c\u3063\u3066\u304f\u3060\u3055\u3044\u3002\u305d\u308c\u305e\u308c\u306b\u4e00\u8a00\u306e\u8aac\u660e\u3092\u4ed8\u3051\u3066\u304f\u3060\u3055\u3044\u3002&quot;,\n      &quot;en&quot;: &quot;Write three haiku on the theme of spring, keeping the 5-7-5 syllable pattern, each with a one-line explanation.&quot;\n    }\n  ]\n}\n<\/code><\/pre>\n<details>\n<summary>Sandbox settings (systemd unit properties, as used)<\/summary>\n<pre><code>User=lmw-sandbox Group=lmw-sandbox Slice=lmw-lab.slice StateDirectory=lmw-sandbox Environment=HOME=\/var\/lib\/lmw-sandbox\/magnitude AllowedCPUs=1-3 MemoryMax=16G TasksMax=512 Nice=10 IOSchedulingClass=idle PrivateNetwork=yes RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 ProtectHome=yes ProtectSystem=strict PrivateTmp=yes PrivateDevices=yes PrivateIPC=yes NoNewPrivileges=yes CapabilityBoundingSet= AmbientCapabilities= RestrictSUIDSGID=yes ProtectKernelTunables=yes ProtectKernelModules=yes ProtectKernelLogs=yes ProtectControlGroups=yes ProtectClock=yes ProtectHostname=yes ProtectProc=invisible LockPersonality=yes RestrictRealtime=yes RestrictNamespaces=yes SystemCallArchitectures=native InaccessiblePaths=-\/run\/docker.sock -\/var\/run\/docker.sock -\/run\/containerd -\/var\/lib\/docker -\/etc\/sudoers -\/etc\/sudoers.d<\/code><\/pre>\n<\/details>\n<p><!-- \/lmw:engine-compare --><\/p>\n<p><!-- lmw:related --><\/p>\n<h2>Related Articles<\/h2>\n<ul>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/06\/magnitudedev-magnitude-026-released\/\">magnitudedev\/magnitude v0.2.6 Released: Major Mac Speedups<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/03\/magnitude-cli-v0-2-5-released\/\">Magnitude CLI v0.2.5 Released: M5 Mac &amp; MoE Speedups<\/a><\/li>\n<li><a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/2026\/10\/03\/magnitude-0-2-4-released\/\">Magnitude 0.2.4 Released: Faster Inference and Lower Memory<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:related --><\/p>\n<p><!-- lmw:next-steps --><\/p>\n<h2>What to Read Next<\/h2>\n<ul>\n<li><strong>Engines mentioned in this article<\/strong> \u2192 <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-magnitude-en\/\">Magnitude<\/a> \/ <a href=\"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/engine-llama-cpp-en\/\">llama.cpp<\/a><\/li>\n<\/ul>\n<p><!-- \/lmw:next-steps --><\/p>\n<h2>Sources<\/h2>\n<ul>\n<li><a href=\"https:\/\/github.com\/magnitudedev\/magnitude\">GitHub &#8211; magnitudedev\/magnitude<\/a><\/li>\n<li><a href=\"https:\/\/news.ycombinator.com\/item?id=49911995\">Hacker News Launch HN: Magnitude (YC S25) \u2013 Self-optimizing inference engine for agents<\/a><\/li>\n<\/ul>\n<p><!-- lmw:updates --><\/p>\n<h2>Update History<\/h2>\n<ul>\n<li>2026-10-02: Added our own measurements: speed compared with llama.cpp on our CPU.<\/li>\n<li>2026-10-02: Added the result of our own measurements to the title.<\/li>\n<\/ul>\n<p><!-- \/lmw:updates --><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Magnitude is an open-source inference engine for AI agents that auto-optimizes kernels for hardware, beating llama.cpp speeds.<\/p>\n","protected":false},"author":1,"featured_media":8766,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[105],"tags":[1073,2791,2793,1547,2795],"class_list":["post-8555","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-engines-and-tools","tag-llama-cpp-en","tag-magnitude-en","tag-rust-en","tag-verified","tag--en"],"lang":"en","translations":{"en":8555,"ja":8553},"_links":{"self":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/8555","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/comments?post=8555"}],"version-history":[{"count":6,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/8555\/revisions"}],"predecessor-version":[{"id":10276,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/posts\/8555\/revisions\/10276"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media\/8766"}],"wp:attachment":[{"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/media?parent=8555"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/categories?post=8555"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/localmodelwatch.tsuchitsuchi.com\/en\/wp-json\/wp\/v2\/tags?post=8555"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}