llmspeed: A Simulator to Estimate Local LLM Speeds

llmspeed: A Simulator to Estimate Local LLM Speeds

At a Glance

Item Value
Published 2026-10-17
License MIT
Source type Unverified (not confirmed by a primary source)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

A new simulator tool called “llmspeed" has been released, allowing users to experience and estimate the operating speed of local LLMs before purchasing hardware, according to reports. Note that this information is based on a submission to Hacker News and contains unverified details that have not been fully officially validated.

llmspeed is an open-source project developed in TypeScript and licensed under the MIT license. By specifying hardware environments such as GPUs for PCs and servers, Macs, and CPUs, along with various open-weight models, users can estimate generation speeds when running short prompts on a single device. It serves as a decision-making tool for engineers considering high-end hardware investments for local LLM deployment, or those who want to gauge expected speeds before downloading massive model files.

Key Changes

Support for Estimating Diverse Hardware Environments

llmspeed allows users to calculate speeds by selecting from a wide range of hardware, from desktop GPUs and Macs to workstation products and cloud GPUs, according to reports. Specifically, the list includes NVIDIA RTX 3060 12GB, RTX 4070 12GB, RTX 4070 Ti Super 16GB, RTX 4090 24GB, RTX 5060 Ti 16GB, RTX 5070 Ti 16GB, RTX 5080 16GB, RTX 5090 32GB, RTX PRO 6000 96GB, and DGX Spark 128GB. AMD products listed include RX 7900 XT 20GB, RX 7900 XTX 24GB, RX 9070 XT 16GB, Radeon AI PRO R9700 32GB, and Ryzen AI Max+ 395 128GB (Strix Halo). Furthermore, Apple Silicon environments feature MacBook M2 Pro 16GB, MacBook M3 Max 48GB, Mac Studio M3 Ultra 512GB, MacBook M4 Pro 24GB, Mac Mini M4 Pro 64GB, Mac Studio M4 Max 128GB, MacBook Pro M5 Pro 48GB, MacBook Pro M5 Max 128GB, and Mac Studio M5 Ultra 512GB, alongside Intel Arc Pro B70 32GB, CPU-only environments (Ryzen 9 9950X, 96GB DDR5), and cloud-focused A100 80GB, H100 80GB, and H200 141GB options. This helps engineers planning to procure equipment to compare and evaluate the processing capacity they can expect from their intended hardware setups.

Comprehensive Coverage of Latest Open-Weight Models

LLMs of various sizes and architectures can be specified as models for estimation. The lineup includes Llama 3.1 8B, Qwen 3.5 9B, Qwen 3.6/3.8 27B, Qwen 3.5/3.6 35B A3B, Qwen3 30B A3B, Ornith 1.5 9B, Ornith 1.5 35B A3B, Gemma 4 26B A4B, Gemma 4 31B, DeepSeek V4 Flash, DeepSeek R1 Distill 14B, GLM-5.3 Flash, GPT-OSS 20B, GPT-OSS 120B, and Muse Glimmer 30B. Users can simply select the model they want to run to check performance estimates tailored to each model architecture and parameter scale.

Integration of Community Measurements and Spec Estimates

Figures displayed in llmspeed are calculated as estimates for short prompts on a single device. It is explained that some measured values are sourced from community benchmarks like r/LocalLLaMA and localscore.ai, while other figures are derived from similar configurations or estimated from hardware specifications. Although the community has pointed out differences from actual operational environments—such as long context lengths (e.g., 60k to 140k tokens), speculative decoding, and multi-GPU configurations—it can be utilized as a way to quickly grasp a rough sense of speed based on model and hardware combinations.

Warning Displays for Insufficient Memory Combinations

If the required memory amount of the selected model exceeds the available hardware memory, a warning is displayed stating that execution is not possible. For example, when selecting Qwen 3.6/3.8 27B in FP16 (required memory 54.6 GB) on an AMD RX 9070 XT 16GB (available memory 14.9 GB), a notification appears prompting the user to change quantization or reselect the model and hardware because the model does not fit. This helps prevent failures such as devices failing to run due to insufficient memory after purchase or download.

Hardware Requirements

The support status of hardware and models that can be selected for estimation in this tool is as follows. Note that this information is based on community submissions and includes unverified information that has not been officially confirmed.

Supported Hardware

llmspeed offers a diverse selection of computing resources, ranging from desktop GPUs to laptops, Macs with large unified memory, and server/cloud GPUs. Aimed at operational estimation in single-device environments, the specifically listed hardware includes:

  • NVIDIA GeForce / Desktop: NVIDIA RTX 3060 12GB, NVIDIA RTX 3090 24GB, NVIDIA RTX 4070 12GB, NVIDIA RTX 4070 Ti Super 16GB, NVIDIA RTX 4090 24GB, NVIDIA RTX 5060 Ti 16GB, NVIDIA RTX 5070 Ti 16GB, NVIDIA RTX 5080 16GB, NVIDIA RTX 5090 32GB
  • NVIDIA Workstation / Systems: NVIDIA RTX PRO 6000 96GB, NVIDIA DGX Spark 128GB
  • Apple Silicon (Mac): MacBook M2 Pro 16GB, MacBook M3 Max 48GB, Mac Studio M3 Ultra 512GB, MacBook M4 Pro 24GB, Mac Mini M4 Pro 64GB, Mac Studio M4 Max 128GB, MacBook Pro M5 Pro 48GB, MacBook Pro M5 Max 128GB, Mac Studio M5 Ultra 512GB
  • AMD GPU / APU: AMD RX 7900 XT 20GB, AMD RX 7900 XTX 24GB, AMD RX 9070 XT 16GB, AMD Radeon AI PRO R9700 32GB, AMD Ryzen AI Max+ 395 128GB (Strix Halo)
  • Intel GPU: Intel Arc Pro B70 32GB
  • CPU-only: CPU-only (Ryzen 9 9950X, 96GB DDR5)
  • Cloud Accelerators: A100 80GB (cloud), H100 80GB (cloud), H200 141GB (cloud)

Supported Models

Open-weight models targeted for estimation include a wide range of architectures, from standard dense models to the latest Mixture of Experts (MoE) models and distilled models.

  • Llama / Gemma Series: Llama 3.1 8B, Gemma 4 26B A4B, Gemma 4 31B
  • Qwen Series: Qwen 3.5 9B, Qwen 3.6/3.8 27B, Qwen 3.5/3.6 35B A3B, Qwen3 30B A3B
  • DeepSeek Series: DeepSeek R1 Distill 14B, DeepSeek V4 Flash
  • GPT-OSS / Ornith / Others: GPT-OSS 20B, GPT-OSS 120B, Ornith 1.5 9B, Ornith 1.5 35B A3B, GLM-5.3 Flash, Muse Glimmer 30B

By combining these hardwares, models, and quantization precisions (such as FP16 or Q4), the tool is reported to estimate generation speed (tok/s) on a single device, Time to first token (TTFT), and the breakdown in seconds for KV cache and Prefill processing steps. For instance, simulation results indicate that when given a 5,687-token prompt in a Mac Studio M4 Max 128GB environment, Gemma 4 31B with Q4 takes 36 seconds for TTFT, whereas Gemma 4 26B A4B with Q4 takes only 4.7 seconds. Additionally, instances have been reported where the TTFT for Gemma 4 31B under the same settings on a MacBook Pro M5 Max 128GB environment is displayed as 19 seconds.

On the other hand, users in the Hacker News community have expressed desires for features such as multi-GPU configurations (identical, hybrid, and with/without NVLink), PCIe generation selection, support for speculative decoding, consideration of long context lengths (such as 60k to 140k tokens), and the addition of older hardware like the GTX 1070 or RX 580.

How to Get It

llmspeed is provided as a web-based tool and can be used by directly accessing the official website below via a web browser:

llmspeed Official Website

You can check estimation results simply by selecting hardware and models on the browser, without needing to set up an inference engine on your local terminal or download large model data.

Furthermore, llmspeed is developed as open-source software written in TypeScript (MIT license), and the source code is available in the GitHub repository (nicorobinreal/llmspeed).

Sources

This article contains unverified information. We will append an update note once it is confirmed by a primary source.