Local Models That Run in 8GB of VRAM (RTX 4060 / 3060 Ti, etc.)
Every model covered by Local Model Watch that fits in 8GB of GPU memory, with the best build that fits. The “Best build for 8GB" column is the largest (highest-quality) quantization or precision whose estimated memory stays within 8GB — often better than the smallest build listed in the VRAM quick reference. Estimates are computed by this site from actual file sizes plus runtime overhead (method); long contexts need more.
17 models listed.
Everything that fits your GPU: 8GB / 12GB / 16GB / 24GB
Contents
Text Generation (Chat and Code)
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 8GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| prism-ml/Ternary-Bonsai-2-27B-gguf | 27.8B | Q1_0 (5.5GB) | 6.6GB | 8GB | Ollama / LM Studio, MLX | Read |
| togethercomputer/Tev1-4B-experimental | 4.7B | Q8_0 (4.3GB) | 5.2GB | 4GB | Ollama / LM Studio, vLLM | Read |
| openbmb/MiniCPM5-2B | 2.5B | F16 (4.7GB) | 5.6GB | 4GB | Ollama / LM Studio, vLLM, MLX | Read |
| harshatheg/Qwen-2.5-1B-RLCD | 1.5B | BF16 (2.9GB) | 3.5GB | 4GB | MLX | Read |
| pfnet/plamo-3-610m-fin-instruct | 890M | F16 (1.7GB) | 2.0GB | 4GB | Ollama / LM Studio, vLLM | Read |
| togethercomputer/Tev1-0.8B-experimental | 873M | F32 (1.6GB) | 2.0GB | 4GB | vLLM | Read |
| LiquidAI/LFM2.5-350M-Diffusion-Exp | 425M | BF16 (0.8GB) | 0.9GB | 4GB | – | Read |
| Cactus-Compute/needle3 | – | Original precision (0.2GB) | 0.3GB | 4GB | – | Read |
Vision-Language and Multimodal Models
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 8GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| Qwen/Qwen3.8-27B | 27.8B | IQ1_M (6.3GB) | 7.5GB | 8GB | Ollama / LM Studio, vLLM, MLX | Read |
| Cloudflare/clef-flash | 9.4B | Q5_K_M (6.4GB) | 7.7GB | 4GB | Ollama / LM Studio, vLLM, MLX | Read |
| apple/LensVLM-9B | 9.4B | Q5_K_M (6.4GB) | 7.7GB | 4GB | Ollama / LM Studio, vLLM, MLX | Read |
| google/DiarizationLM-Gemma-4-E4B-v1 | 8.0B | Q4_K_M (4.9GB) | 5.9GB | 8GB | Ollama / LM Studio, vLLM | Read |
| LiquidAI/LFM2.5-VL-3B-DSpark | 279M | F16 (0.5GB) | 0.6GB | 4GB | – | Read |
Audio: Speech Synthesis, Music and Speech Recognition
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 8GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| phasefield-audio/Irodori-TTS-v4.1-Anime | 766M | F32 (2.9GB) | 3.4GB | 4GB | – | Read |
| FermionResearch/Phonon-2 | 627M | Q8_0 (0.7GB) | 0.8GB | 4GB | – | Read |