Local Models That Run in 24GB of VRAM (RTX 4090 / 3090, etc.)
Every model covered by Local Model Watch that fits in 24GB of GPU memory, with the best build that fits. The “Best build for 24GB" column is the largest (highest-quality) quantization or precision whose estimated memory stays within 24GB — often better than the smallest build listed in the VRAM quick reference. Estimates are computed by this site from actual file sizes plus runtime overhead (method); long contexts need more.
37 models listed.
Everything that fits your GPU: 8GB / 12GB / 16GB / 24GB
Contents
Text Generation (Chat and Code)
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 24GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| Edge0/Edge0-35B-A3B-preview | 36.0B | MLX 4bit (18.2GB) | 21.8GB | 24GB | vLLM, MLX | Read |
| nex-agi/Nex-N2.5-mini | 35.1B | Q4_K_S (19.5GB) | 23.4GB | 16GB | Ollama / LM Studio, vLLM, MLX | Read |
| XingChen-AGI/Xing4.0-29B-A4B | 31.2B | IQ4_NL (18.7GB) | 22.5GB | 24GB | Ollama / LM Studio, MLX | Read |
| orcarouter/OrcaSAQ-2-27B | 27.8B | EXL3 (11.4GB) | 13.7GB | 16GB | vLLM | Read |
| prism-ml/Ternary-Bonsai-2-27B-gguf | 27.8B | Q2_0 (6.7GB) | 8.1GB | 8GB | Ollama / LM Studio, MLX | Read |
| Altworld/Hemmingway-1 | 26.9B | Q5_K_M (19.5GB) | 23.4GB | 12GB | Ollama / LM Studio, vLLM, MLX | Read |
| bartowski/Gryphe_Pantheon-Reasoning-26B-A4B-1.1-V2-GGUF | 26.5B | Q5_K_M (19.4GB) | 23.2GB | 12GB | Ollama / LM Studio | Read |
| EleutherAI/olmo3-7b-sdf-sft-clean150 | 7.3B | BF16 (13.6GB) | 16.3GB | 24GB | vLLM | Read |
| togethercomputer/Tev1-4B-experimental | 4.7B | BF16 (8.1GB) | 9.7GB | 4GB | Ollama / LM Studio, vLLM | Read |
| tencent/Simple-Attention-Sparsification | 4.0B | BF16 (7.5GB) | 9.0GB | 12GB | – | Read |
| openbmb/MiniCPM5-2B | 2.5B | F16 (4.7GB) | 5.6GB | 4GB | Ollama / LM Studio, vLLM, MLX | Read |
| harshatheg/Qwen-2.5-1B-RLCD | 1.5B | BF16 (2.9GB) | 3.5GB | 4GB | MLX | Read |
| pfnet/plamo-3-610m-fin-instruct | 890M | F16 (1.7GB) | 2.0GB | 4GB | Ollama / LM Studio, vLLM | Read |
| togethercomputer/Tev1-0.8B-experimental | 873M | F32 (1.6GB) | 2.0GB | 4GB | vLLM | Read |
| LiquidAI/LFM2.5-350M-Diffusion-Exp | 425M | BF16 (0.8GB) | 0.9GB | 4GB | – | Read |
| Cactus-Compute/needle3 | – | Original precision (0.2GB) | 0.3GB | 4GB | – | Read |
Vision-Language and Multimodal Models
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 24GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| Qwen/Qwen3.8-27B | 27.8B | Q5_K_XL (19.4GB) | 23.3GB | 8GB | Ollama / LM Studio, vLLM, MLX | Read |
| bartowski/vectionlabs_Salience-27B-R6-GGUF | 27.8B | Q5_K_M (19.5GB) | 23.4GB | 12GB | Ollama / LM Studio | Read |
| agentionai/Signal-3.8-27B-GGUF | 27.8B | Q5_K_M (18.2GB) | 21.8GB | 16GB | Ollama / LM Studio | Read |
| Jackrong/Qwopus3.8-27B-Flash-GGUF | 27.8B | Q5_K_M (18.2GB) | 21.8GB | 16GB | Ollama / LM Studio | Read (Japanese) |
| Cloudflare/clef | 27.4B | Q5_K_M (19.3GB) | 23.1GB | 12GB | Ollama / LM Studio, vLLM, MLX | Read |
| ggml-org/OpenJev-GGUF | 27.4B | Q4_K_M (17.7GB) | 21.2GB | 24GB | Ollama / LM Studio | Read |
| bartowski/TheDrummer_Orion-26B-A4B-v1.1-GGUF | 25.8B | Q5_K_M (18.6GB) | 22.4GB | 12GB | Ollama / LM Studio | Read |
| Cloudflare/clef-flash | 9.4B | BF16 (16.7GB) | 20.0GB | 4GB | Ollama / LM Studio, vLLM, MLX | Read |
| apple/LensVLM-9B | 9.4B | BF16 (16.7GB) | 20.0GB | 4GB | Ollama / LM Studio, vLLM, MLX | Read |
| ggml-org/MiMo-V2.6-Distill-Qwen-9B-GGUF | 9.4B | Q8_0 (8.9GB) | 10.6GB | 12GB | Ollama / LM Studio | Read |
| google/DiarizationLM-Gemma-4-E4B-v1 | 8.0B | Q4_K_M (4.9GB) | 5.9GB | 8GB | Ollama / LM Studio, vLLM | Read |
| LiquidAI/LFM2.5-VL-3B-DSpark | 279M | F16 (0.5GB) | 0.6GB | 4GB | – | Read |
| inclusionAI/Realtime-Venus | – | Original precision(Realtime-Venus-Audio) (17.5GB) | 20.9GB | 24GB | – | Read |
Audio: Speech Synthesis, Music and Speech Recognition
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 24GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| Edge0/Audio8-ASR-Infinite | 4.1B | BF16 (7.6GB) | 9.1GB | 12GB | – | Read |
| m-a-p/YuE2-3B | 3.6B | BF16 (6.8GB) | 8.1GB | 12GB | – | Read |
| phasefield-audio/Irodori-TTS-v4.1-Anime | 766M | F32 (2.9GB) | 3.4GB | 4GB | – | Read |
| FermionResearch/Phonon-2 | 627M | Q8_0 (0.7GB) | 0.8GB | 4GB | – | Read |
Task not declared
→ Scroll horizontally to see all columns
| Model | Parameters | Best build for 24GB | Est. memory | Smallest tier | Runs on | Article |
|---|---|---|---|---|---|---|
| nvidia/PixelUMM | 8.2B | BF16 (15.3GB) | 18.3GB | 24GB | – | Read |
| tencent/WeVisDoc-4B | 4.4B | F16 (8.2GB) | 9.9GB | 4GB | – | Read |
| Comfy-Org/YuE2 | 3.6B | BF16 (6.8GB) | 8.1GB | 12GB | – | Read |
| Efficient-Large-Model/H3-to-LTX-Latent-Adapter | 195M | BF16 (0.4GB) | 0.4GB | 4GB | – | Read |