Magnitude: Up to 2x Faster Than llama.cpp Officially, 21x Slower on Our CPU

October 6, 2026

Magnitude: Up to 2x Faster Than llama.cpp Officially, 21x Slower on Our CPU

At a Glance

Item Value
Published 2026-10-01
License Apache-2.0
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

What we checked ourselves

  • On our CPU-only server (Neoverse-N1, 3 cores) with the same model file (Qwen3.5-4B-UD-Q4_K_XL.gguf), Magnitude 0.2.3 generated 0.24–0.41 tokens/s against 4.92–6.19 tokens/s for llama.cpp b11223: about 21× slower (median over our five questions).
  • We do not recommend Magnitude for CPU-only machines. The official “up to 2× faster than llama.cpp” figures are for Metal and CUDA.

The setup, conditions and the exact steps we ran are in “Our Own Measurements” below.

Overview

The initial release of Magnitude has been announced. Magnitude is an open-source inference engine (Apache-2.0 license) developed in Rust specifically for AI agents, featuring the ability to automatically optimize kernels according to the execution hardware.

In this version, device-level kernel compilation and tuning achieve up to 2x faster inference speeds compared to llama.cpp, making it a very powerful choice for users looking to dramatically accelerate agent performance in local environments.

Key Changes

Introduction of Automatic Device-Specific Kernel Optimization

Unlike typical inference engines that distribute pre-compiled kernels for a broad range of hardware, Magnitude compiles and tunes kernels at runtime to match the user’s actual device.

This process generates code tailored perfectly to the characteristics of a specific chip. Whether using Apple Silicon, NVIDIA GPUs, AMD GPUs, or CPU-only environments, users can push their hardware’s native performance to the limit. Because tuning occurs before running a model, an optimization process runs upon initial startup, but subsequent inference speeds improve dramatically. This provides a particularly massive benefit to users with custom hardware configurations where existing general-purpose engines failed to unlock full performance.

Achieving Inference Performance That Outperforms llama.cpp

In benchmarks, it has recorded scores significantly higher than existing engines like llama.cpp. Specifically, a 92% improvement in decoding speed has been confirmed in Metal (Apple Silicon) environments, along with a 19% speedup in CUDA (NVIDIA) environments.

This performance boost directly improves the user experience, especially in agent tasks where sequential token generation is critical. Because open-weight models can be run at speeds unattainable by general-purpose engines, more complex reasoning can be completed in less time. After installation, this high-speed inference environment can be utilized simply by selecting a recommended model from the “Discover" section within the app.

Memory Optimization and Cache Sharing for Parallel Agent Execution

Advanced memory management features have been implemented assuming workloads where multiple agents run simultaneously. Memory usage per agent has been reduced by 27% compared to conventional levels, and memory is released immediately once an agent finishes its task.

Additionally, to enable fast parallel sessions, it features a function to share the prefix cache between sessions. This suppresses duplicate memory consumption and prevents processing stalls even when launching multiple agents using the same system prompt or background knowledge. For developers running multiple agents locally at the same time, this significantly reduces the burden of resource management.

Seamless Integration with Major Agents

One-click connection features have been introduced for widely used open-weight model-based agents such as Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.

Users already utilizing these agents can benefit from a high-speed inference environment without configuration hassles simply by connecting through Magnitude’s “Connections" settings. Furthermore, because it provides an OpenAI-compatible API, it can also be easily used from other custom agents and tools through a standard interface.

Providing a Fully Local and Private Execution Environment

Magnitude is designed with privacy in mind, keeping all prompts, files, and models within the user’s local machine. No internet connection is required except when downloading models.

Not only are token fees avoided, but there is no concern about data being sent to external servers even when operating agents handling sensitive data, making it an optimal solution for engineers looking to build a secure development environment. No special external service registration is required to get started; simply installing the desktop app makes all features, including the CLI, available.

Specifications

Magnitude provides an optimized execution environment for a wide variety of hardware setups and major open-weight models.

On the hardware front, there are no fixed minimum spec requirements for operation. It supports diverse graphic accelerators including Apple Silicon (Mac), NVIDIA GPUs, and AMD GPUs, and can run without issues in CPU-only environments lacking a GPU. Because it operates flexibly according to the machine resources of the execution environment, users can carry out optimal operations suited to each environment, such as running smaller models on low-memory compact machines and larger models on advanced machines equipped with large memory capacities.

Regarding model support, rather than a one-size-fits-all approach covering every architecture, it focuses on open-weight model families that have gained particular popularity in the developer community. By hand-writing dedicated, manually optimized kernels specifically for these models, it achieves inference speeds that vastly surpass general-purpose inference engines. The complete list of specifically supported models is progressively checked and updated on the official website ( magnitude.dev/models ).

Furthermore, to accommodate AI agent use cases, connectivity with a broad ecosystem is secured. Supported agents that can be easily linked with a single click of the operation include Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Even for third-party agents or custom-developed tools not included in the list, communication can be conducted via standard OpenAI-compatible APIs to similarly leverage Magnitude’s high-speed inference engine.

How to Get It

Magnitude’s setup and onboarding steps are organized to be extremely simple, provided as an integrated application supporting major desktop OSs.

Specific steps from installation to starting usage are as follows:

  1. Download and Launch the Desktop App
    Download Magnitude provided for macOS, Windows, and Linux operating systems, install it on your environment, and launch the application. Because this desktop app bundles the magnitude CLI from the start, there is zero hassle of individually installing and configuring extra CLI tools for terminal operations.

  2. Select and Download a Model
    Open the “Discover" section in the app navigation. Here, select an open-weight model matching your hardware environment and use case from the recommended models, and download it to your local environment.

  3. Connect Agents and Start Execution
    Go to the “Connections" section in the app, select the agent you normally use (Pi, OpenCode, Hermes, Codex, etc.), and establish a connection. Because linking setup completes with a single click, you can immediately start inference processing through the agent.

Once model downloading is complete, all subsequent prompt processing, file referencing, and model execution happen entirely within the local machine. Because zero data transmission to external networks occurs, you can immediately start setup and operations in a private and high-speed environment.

Our Own Measurements

Magnitude vs llama.cpp on a CPU Only

Magnitude publishes “up to 2× faster than llama.cpp”, but those figures are for Metal (Apple Silicon) and CUDA (NVIDIA); no CPU-only figures are published. We ran it on our server, which has no GPU, gave llama.cpp the same model file, and sent both the same questions in the same way.

We do not recommend Magnitude for CPU-only machines. Under these conditions its generation was about 21× slower than llama.cpp (15–22× per question), and prompt processing 16–38× slower. If you run models on a CPU, llama.cpp (or a tool built on it) was by far the faster choice here.

Item Condition
CPU Neoverse-N1 (no GPU), 3 cores
Model Qwen3.5-4B-UD-Q4_K_XL.gguf from unsloth/Qwen3.5-4B-MTP-GGUF (revision 86835bf9), the file Magnitude’s catalog uses for qwen3.5-4b:gguf:q4; both engines read the same file
Magnitude 0.2.3 (magnitude-inference, native build 0.1.0+src.3cc3838e83767a1f232f223b). Context fixed at 262,144 tokens; MTP speculative decoding on (automatic)
llama.cpp b11223 (official ubuntu-arm64 build), context 4,096 tokens; with and without MTP speculative decoding (--spec-type draft-mtp)
Questions Our five fixed Japanese questions (version 2), temperature 0, up to 512 tokens, thinking off, streaming
Date 2026-10-01

→ Scroll horizontally to see all columns

Question Magnitude generation llama.cpp generation llama.cpp + MTP generation Magnitude: time per answer llama.cpp: time per answer
Summarization 0.24 tok/s 4.92 tok/s 4.83 tok/s 33 min 67 s
Formal business email 0.25 tok/s 5.68 tok/s 5.04 tok/s 28 min 93 s
Word problem 0.41 tok/s 5.99 tok/s 6.64 tok/s 18 min 89 s
Knowledge 0.31 tok/s 5.72 tok/s 6.27 tok/s 9 min 92 s
Constrained writing 0.29 tok/s 6.19 tok/s 5.08 tok/s 11 min 38 s

Other things we observed:

  • Prompt processing: Magnitude 0.49–0.79 tok/s, llama.cpp 12.91–18.65 tok/s.
  • Preparing to load: every load first compiled and tuned kernels for this CPU, which took 33–48 minutes across our 3 loads.
  • Memory: the context is fixed at the model’s maximum (262,144 tokens) and the load plan asked for 9.9GB. Loads with a 12 GB and 14 GB memory limit were refused for lack of memory after the preparation finished; with 16GB it loaded (peak 12.2GB).
  • Streaming: even with streaming requested, Magnitude’s text arrived all at once at the end of each answer (first text after 9–33 minutes); llama.cpp’s first text came after 2–39 seconds.
  • Magnitude’s MTP accepted 12%–45% of the drafted tokens.

Please note:

  • These are figures from one CPU-only ARM server. They are not a guide to speed on a Mac (Metal) or an NVIDIA GPU, and they do not contradict the official figures for those.
  • Magnitude is normally used through its desktop app or magnitude serve; we started the component that does the inference in those, magnitude-inference, on its own.
  • We ran Magnitude (a prebuilt binary) isolated from the rest of our server (a dedicated user, no network, 3 CPU cores) and ran llama.cpp in the same sandbox.
  • We compared speed only, not the content of the answers.

Setup, Download and Run Steps (What We Actually Ran)

Run these in an empty working folder. icn/installation.json is a file we wrote ourselves (its format comes from inference/service/server/src/installation.rs in the source). We ran each python3 bench.py inside the sandbox described above (the settings are listed at the end).

# 1. Download Magnitude 0.2.3's inference component (Linux ARM64), check its SHA-256 against the release record
curl -LO 'https://github.com/magnitudedev/magnitude/releases/download/%40magnitudedev%2Fcli%400.2.3/magnitude-release.json'
curl -LO 'https://github.com/magnitudedev/magnitude/releases/download/%40magnitudedev%2Fcli%400.2.3/magnitude-icn-base-linux-arm64-gnu.tar.gz'
echo 'b0824a230b1b90542a3a4d2d95d278d5f655ef3305b918b88964e1e48ed34c1a  magnitude-icn-base-linux-arm64-gnu.tar.gz' | sha256sum -c -
mkdir -p icn && tar xzf magnitude-icn-base-linux-arm64-gnu.tar.gz -C icn
echo '{"schemaVersion":1,"nativeBuild":"0.1.0+src.3cc3838e83767a1f232f223b"}' > icn/installation.json
mkdir -p store cache out

# 2. llama.cpp b11223 (the official ubuntu-arm64 build)
curl -LO 'https://github.com/ggml-org/llama.cpp/releases/download/b11223/llama-b11223-bin-ubuntu-arm64.tar.gz'
echo 'e4280e5c71369a04bef5a171e50ab77fd923f6c86318106b19942397c4ded75f  llama-b11223-bin-ubuntu-arm64.tar.gz' | sha256sum -c -
tar xzf llama-b11223-bin-ubuntu-arm64.tar.gz && mv llama-b11223 llama

# 3. Place the model (the revision pinned by Magnitude's catalog) in Hugging Face cache layout
REV=86835bf9949e4d14d6860f7910b1340ad4f271a9
H=hf/hub/models--unsloth--Qwen3.5-4B-MTP-GGUF
mkdir -p $H/blobs $H/snapshots/$REV $H/refs && echo -n $REV > $H/refs/main
curl -s "https://huggingface.co/api/models/unsloth/Qwen3.5-4B-MTP-GGUF/revision/$REV?blobs=true" | python3 -c "
import json,sys;d=json.load(sys.stdin)
for s in d['siblings']:
  if s['rfilename'] in ('Qwen3.5-4B-UD-Q4_K_XL.gguf','mmproj-BF16.gguf'): print(s['rfilename'], s['lfs']['sha256'], s['size'])" > files.txt
while read name sha size; do
  curl -sfL -o $H/blobs/$sha "https://huggingface.co/unsloth/Qwen3.5-4B-MTP-GGUF/resolve/$REV/$name"
  echo "$sha  $H/blobs/$sha" | sha256sum -c -
  ln -sfn ../../blobs/$sha $H/snapshots/$REV/$name
done < files.txt

# 4. Send the same questions to the three setups (bench.py below)
python3 bench.py magnitude 2 512
python3 bench.py llama 1 512
python3 bench.py llama-mtp 1 512

bench.py (written by us, in full) starts each engine, sends the five questions with streaming, and records the speeds each engine reports. Files go under the working folder:

"""Magnitude と llama-server に同じ質問を同じ方法で送り、速さを比べる(箱の中で動かす)。"""
import json, subprocess, sys, time, urllib.request, os
M = os.path.abspath(".")
GGUF = f"{M}/hf/hub/models--unsloth--Qwen3.5-4B-MTP-GGUF/snapshots/86835bf9949e4d14d6860f7910b1340ad4f271a9/Qwen3.5-4B-UD-Q4_K_XL.gguf"
engine = sys.argv[1]
rounds = int(sys.argv[2]) if len(sys.argv) > 2 else 2
max_tokens = int(sys.argv[3]) if len(sys.argv) > 3 else 512
PORT = 18080
if engine == "magnitude":
    cmd = [f"{M}/icn/bin/magnitude-inference", "serve", "--bind", f"127.0.0.1:{PORT}",
           "--model-store", f"{M}/store", "--cache-root", f"{M}/cache",
           "--hf-cache", f"{M}/hf/hub", "--installation", f"{M}/icn/installation.json"]
    env = dict(os.environ)
else:
    b = f"{M}/llama"
    cmd = [f"{b}/llama-server", "-m", GGUF, "-t", "3", "-c", "4096", "--jinja", "--no-mmproj",
           "--host", "127.0.0.1", "--port", str(PORT)]
    if engine == "llama-mtp":
        cmd += ["--spec-type", "draft-mtp"]
    env = dict(os.environ, LD_LIBRARY_PATH=b)
log = open(f"{M}/out/{engine}.log", "w")
p = subprocess.Popen(cmd, stdout=log, stderr=subprocess.STDOUT, env=env)

def call(path, data=None, timeout=60):
    req = urllib.request.Request(f"http://127.0.0.1:{PORT}{path}",
                                 data=json.dumps(data).encode() if data is not None else None,
                                 headers={"Content-Type": "application/json"},
                                 method="POST" if data is not None else "GET")
    with urllib.request.urlopen(req, timeout=timeout) as r:
        return json.loads(r.read() or b"null")

def ask(model, text):
    body = {"model": model, "messages": [{"role": "user", "content": text}], "temperature": 0, "seed": 0,
            "max_tokens": max_tokens, "stream": True, "stream_options": {"include_usage": True}}
    if engine == "magnitude":
        body["reasoning_effort"] = "none"
    else:
        body["chat_template_kwargs"] = {"enable_thinking": False}
    req = urllib.request.Request(f"http://127.0.0.1:{PORT}/v1/chat/completions", data=json.dumps(body).encode(),
                                 headers={"Content-Type": "application/json"}, method="POST")
    t0 = time.time(); first = None; last = None; chunks = 0; usage = None; timings = None; text_out = []; finish = None
    with urllib.request.urlopen(req, timeout=3600) as r:
        for raw in r:
            line = raw.decode("utf-8", "replace").strip()
            if not line.startswith("data:"):
                continue
            payload = line[5:].strip()
            if payload == "[DONE]":
                break
            ev = json.loads(payload)
            usage = ev.get("usage") or usage
            timings = ev.get("timings") or timings
            for ch in ev.get("choices") or []:
                d = ch.get("delta") or {}
                piece = (d.get("content") or "") + (d.get("reasoning_content") or d.get("reasoning") or "")
                finish = ch.get("finish_reason") or finish
                if piece:
                    now = time.time()
                    first = first or now
                    last = now
                    chunks += 1
                    text_out.append(d.get("content") or "")
    n = (usage or {}).get("completion_tokens") or chunks
    res = {"ttft_s": round(first - t0, 3) if first else None,
           "completion_tokens": n, "chunks": chunks, "finish": finish,
           "prompt_tokens": (usage or {}).get("prompt_tokens"),
           "decode_tps": round((n - 1) / (last - first), 2) if first and last and last > first else None,
           "wall_s": round(time.time() - t0, 2), "head": "".join(text_out)[:80]}
    if timings:
        res["native"] = {k: timings.get(k) for k in ("prompt_n", "prompt_per_second", "predicted_n", "predicted_per_second", "draft_n", "draft_n_accepted")}
    return res

out = {"engine": engine, "rounds": []}
try:
    t_start = time.time()
    for _ in range(600):
        if p.poll() is not None:
            raise SystemExit("server exited")
        try:
            if call("/health", timeout=5): break
        except Exception:
            time.sleep(1)
    if engine == "magnitude":
        model = "qwen3.5-4b:gguf:q4"
        try: print("plan", json.dumps(call(f"/api/v1/models/{model}/load-plan", {}), ensure_ascii=False)[:1500], flush=True)
        except Exception as e: print("plan ERR", e, getattr(e, "read", lambda: b"")()[:800], flush=True)
        t = time.time()
        inst = call("/api/v1/instances", {"modelId": model}, timeout=7200)
        out["load_s"] = round(time.time() - t, 1)
        print("instance", json.dumps(inst, ensure_ascii=False)[:1500], flush=True)
    else:
        model = "qwen"
    out["ready_s"] = round(time.time() - t_start, 1)
    prompts = json.load(open(f"{M}/prompts.json"))["prompts"]
    for r in range(rounds):
        rows = []
        for pr in prompts:
            res = ask(model, pr["ja"]); res["id"] = pr["id"]; rows.append(res)
            print(r, json.dumps(res, ensure_ascii=False), flush=True)
        out["rounds"].append(rows)
finally:
    p.terminate()
    try: p.wait(60)
    except Exception: p.kill()
    json.dump(out, open(f"{M}/out/{engine}.json", "w"), ensure_ascii=False, indent=1)

prompts.json placed in the working folder (our fixed questions, in full):

{
  "version": 2,
  "note": "当サイトでの実測(lmw/lab/cpu_bench.py)で全モデルに同じ条件で尋ねる日本語の質問。変えるときは version を上げる(過去の回答と比べられなくなるため、実測は取り直しになる)。reference は答えが1つに決まる質問の正解で、人間が確かめた事実だけを書く。採点には使わず、読者が見比べるために表示する。 2026-09-28 v2: 要約の元の文章を短い段落(約200字)から書き下ろしの記事(約1,200字)に替えた(ユーザーの指摘: 短すぎて要約できたか判断できない)。",
  "prompts": [
    {
      "id": "summary",
      "label": {
        "ja": "要約",
        "en": "Summarization"
      },
      "ja": "次の記事を3文で要約してください。\n\nみどり市は今年の春、市内に三つある公立図書館のうち最も古い中央図書館の改修を終え、十月一日に再開館した。改修は約一年半にわたり、総事業費は約十二億円。このうち約四億円は国の補助金でまかなわれた。\n\n最大の変化は、閲覧席の配置である。以前は館内の座席の大半が学習用の机で、試験期間になると中高生で埋まり、高齢の利用者や小さな子ども連れが座れないという苦情が毎年寄せられていた。改修後は座席数を百二十席から二百席に増やしたうえで、学習用の机を三階にまとめ、一階は会話ができる「にぎわいフロア」、二階は静かに本を読む「閲覧フロア」に分けた。一階には予約制のグループ学習室が四室と、飲み物を持ち込めるカフェスペースも設けられた。\n\n蔵書の面でも見直しが進んだ。貸出の少ない古い資料約三万冊を市内の他の館や保存書庫に移し、代わりに子ども向けの絵本と、介護・子育て・仕事探しといった暮らしに関わる実用書を約一万五千冊増やした。電子書籍の貸出サービスも始まり、利用登録をすれば自宅から一度に三冊まで、二週間借りられる。\n\n建物そのものにも手が入った。築四十年を超えていた空調設備を入れ替え、屋上には太陽光パネルを設置して、館内で使う電力の約二割をまかなえるようにした。車いすの利用者から要望の多かったエレベーターは一基から二基に増え、書架の間の通路も以前より三十センチ広げられた。入口には段差のないスロープが新たに設けられ、授乳室と多目的トイレも各階に一つずつ置かれた。\n\n一方で課題も残る。駐車場は改修前と同じ四十台のままで、週末には近くの道路に順番待ちの車が並んだ。市は隣接する市有地を臨時駐車場として開放したが、恒久的な対策は決まっていない。また、開館時間を平日の夜九時まで延ばしたことで職員の負担が増えており、職員団体は人員の追加を求めている。\n\n利用者の受け止めはおおむね好意的だ。週に二度、孫を連れて来るという七十代の女性は「前は子どもが声を出すと周りの目が気になったが、一階なら気兼ねなく絵本を読んであげられる」と話す。一方、受験を控えた高校生の中には「学習席が三階だけになり、夕方はすぐに満席になる」と不満を漏らす人もいる。図書館は、学習席の空き状況をホームページで確認できる仕組みを年内に導入する予定だ。\n\n再開館に合わせて、催しも増えた。毎週土曜の午前には読み聞かせ会、月に一度は地元の農家や商店主を講師に招く「まちの先生」講座が開かれる。十一月には、市内の中学生が選んだ本を紹介する展示も予定されている。\n\n館長の山本和子さんは「本を借りる場所から、人が集まって過ごす場所へ変えたかった。再開館からの一か月で来館者は以前の一・八倍になった」と話す。市は来年度、残る二つの図書館についても利用者への聞き取りを行い、改修の必要性を検討する方針だ。",
      "en": "Summarize the following article in three sentences. (The article, about 1,200 Japanese characters, describes the renovation and reopening of a city's central library: the budget, the new floor layout and seating, changes to the collection and e-book lending, building and accessibility work, remaining issues such as parking and staff workload, users' reactions, new events, and the city's next steps.)"
    },
    {
      "id": "keigo",
      "label": {
        "ja": "敬語のメール",
        "en": "Formal business email"
      },
      "ja": "取引先の担当者に、来週火曜日の打ち合わせを水曜日に変更してほしいとお願いするメールを、丁寧な敬語で書いてください。",
      "en": "Write an email in polite Japanese (keigo) asking a client's contact person to move next Tuesday's meeting to Wednesday."
    },
    {
      "id": "arithmetic",
      "label": {
        "ja": "文章題",
        "en": "Word problem"
      },
      "ja": "りんごが1個120円、みかんが1個45円です。りんごを3個とみかんを8個買って、1,000円札で払いました。おつりはいくらですか。考え方も示してください。",
      "en": "Apples cost 120 yen each and mandarins cost 45 yen each. You buy 3 apples and 8 mandarins and pay with a 1,000-yen bill. How much change do you get? Show your reasoning.",
      "reference": {
        "ja": "正解は280円です(120×3=360円、45×8=360円、1,000−720=280円)。",
        "en": "The correct answer is 280 yen (120×3 = 360, 45×8 = 360, 1,000 − 720 = 280)."
      }
    },
    {
      "id": "prefectures",
      "label": {
        "ja": "知識",
        "en": "Knowledge"
      },
      "ja": "日本の都道府県のうち、海に面していない県をすべて挙げてください。",
      "en": "List all of Japan's prefectures that do not border the sea.",
      "reference": {
        "ja": "正解は8県です(栃木県・群馬県・埼玉県・山梨県・長野県・岐阜県・滋賀県・奈良県)。",
        "en": "The correct answer is eight prefectures: Tochigi, Gunma, Saitama, Yamanashi, Nagano, Gifu, Shiga and Nara."
      }
    },
    {
      "id": "haiku",
      "label": {
        "ja": "指示どおりの創作",
        "en": "Constrained writing"
      },
      "ja": "「春」をテーマにした俳句を3つ、五・七・五の音数を守って作ってください。それぞれに一言の説明を付けてください。",
      "en": "Write three haiku on the theme of spring, keeping the 5-7-5 syllable pattern, each with a one-line explanation."
    }
  ]
}
Sandbox settings (systemd unit properties, as used)
User=lmw-sandbox Group=lmw-sandbox Slice=lmw-lab.slice StateDirectory=lmw-sandbox Environment=HOME=/var/lib/lmw-sandbox/magnitude AllowedCPUs=1-3 MemoryMax=16G TasksMax=512 Nice=10 IOSchedulingClass=idle PrivateNetwork=yes RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 ProtectHome=yes ProtectSystem=strict PrivateTmp=yes PrivateDevices=yes PrivateIPC=yes NoNewPrivileges=yes CapabilityBoundingSet= AmbientCapabilities= RestrictSUIDSGID=yes ProtectKernelTunables=yes ProtectKernelModules=yes ProtectKernelLogs=yes ProtectControlGroups=yes ProtectClock=yes ProtectHostname=yes ProtectProc=invisible LockPersonality=yes RestrictRealtime=yes RestrictNamespaces=yes SystemCallArchitectures=native InaccessiblePaths=-/run/docker.sock -/var/run/docker.sock -/run/containerd -/var/lib/docker -/etc/sudoers -/etc/sudoers.d

Related Articles

What to Read Next

Sources

Update History

  • 2026-10-02: Added our own measurements: speed compared with llama.cpp on our CPU.
  • 2026-10-02: Added the result of our own measurements to the title.