UPDATED FOR DEEPSEEK-R1, QWEN3, LLAMA 4 & RTX 5090
LLM VRAM, KV-Cache & Token Speed Calculator
Calculate exact GPU memory requirements (Weights + GQA/MLA KV Cache + Activation Buffer) and generation speed (tokens/sec) across 36 open-source LLMs and 23 GPUs / Mac Unified Memory configs.
⚡ Quick Model × Hardware Matchmaker
🔥 Most Searched LLM × GPU Compatibility Benchmarks
FITS (Q4_K_M)
~31.4 tok/s
DeepSeek-R1-Distill-Qwen-32B
on NVIDIA RTX 4090 (24GB)
Requires 23.14 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~31.4 tok/s
Qwen3-32B
on NVIDIA RTX 4090 (24GB)
Requires 23.14 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~92.6 tok/s
DeepSeek-R1 671B (MoE)
on 8x NVIDIA H100 80GB (640GB) (640GB)
Requires 420.2 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~24.8 tok/s
DeepSeek-R1 671B (MoE)
on Apple M3 Ultra Mac Studio (512GB) (512GB)
Requires 420.2 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~13.6 tok/s
Llama-3.3-70B-Instruct
on 2x NVIDIA RTX 3090 (48GB) (48GB)
Requires 47.25 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~29.5 tok/s
Llama-3.3-70B-Instruct
on NVIDIA A100 80GB SXM (80GB)
Requires 47.25 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~280 tok/s
Qwen3-30B-A3B (MoE)
on NVIDIA RTX 4090 (24GB)
Requires 20.45 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~55.9 tok/s
Qwen2.5-Coder-32B-Instruct
on NVIDIA RTX 5090 (32GB)
Requires 23.14 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~37.6 tok/s
Google Gemma-3-27B-IT
on NVIDIA RTX 4090 (24GB)
Requires 21.64 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~122.7 tok/s
Llama-4-Scout-17B-16E (109B MoE)
on NVIDIA A100 80GB SXM (80GB)
Requires 70.22 GB VRAM at 8K ctx →
OVERFLOW
~4 tok/s
Mistral-Small-3.1-24B-Instruct
on NVIDIA RTX 4080 Super (16GB)
Requires 16.9 GB VRAM at 8K ctx →
FITS (Q4_K_M)
~24.9 tok/s
Qwen3-14B
on NVIDIA RTX 3060 12GB (12GB)
Requires 11.15 GB VRAM at 8K ctx →
📊 Complete LLM VRAM Reference Table (8K Context, Batch=1)
| Model Name | Params (Active) | Architecture | FP16 VRAM | FP8 / Q8_0 | Q4_K_M (4-bit) | Min GPU (4-bit) |
|---|---|---|---|---|---|---|
| DeepSeek-R1 671B (MoE) | 671B (37B) | MLA-MoE | 1383.46 GB | 735.52 GB | 420.2 GB | Apple M3 Ultra Mac Studio (512GB) (512GB) |
| DeepSeek-V3 671B (MoE) | 671B (37B) | MLA-MoE | 1383.46 GB | 735.52 GB | 420.2 GB | Apple M3 Ultra Mac Studio (512GB) (512GB) |
| DeepSeek-R1-Distill-Llama-70B | 70.6B (70.6B) | Dense-GQA | 148.6 GB | 80.43 GB | 47.25 GB | NVIDIA RTX 6000 Ada (48GB) |
| DeepSeek-R1-Distill-Qwen-32B | 32.8B (32.8B) | Dense-GQA | 70.23 GB | 38.55 GB | 23.14 GB | NVIDIA RTX 4090 (24GB) |
| DeepSeek-R1-Distill-Qwen-14B | 14.8B (14.8B) | Dense-GQA | 32.65 GB | 18.36 GB | 11.4 GB | NVIDIA RTX 3060 12GB (12GB) |
| DeepSeek-R1-Distill-Llama-8B | 8.03B (8.03B) | Dense-GQA | 18.2 GB | 10.45 GB | 6.67 GB | NVIDIA RTX 3060 12GB (12GB) |
| DeepSeek-R1-Distill-Qwen-7B | 7.62B (7.62B) | Dense-GQA | 16.79 GB | 9.43 GB | 5.85 GB | NVIDIA RTX 3060 12GB (12GB) |
| Qwen3-235B-A22B (MoE) | 235B (22B) | MoE-GQA | 486.23 GB | 259.3 GB | 148.87 GB | NVIDIA Blackwell B200 192GB (192GB) |
| Qwen3-30B-A3B (MoE) | 30.5B (3.3B) | MoE-GQA | 64.23 GB | 34.78 GB | 20.45 GB | NVIDIA RTX 4090 (24GB) |
| Qwen3-32B | 32.8B (32.8B) | Dense-GQA | 70.23 GB | 38.55 GB | 23.14 GB | NVIDIA RTX 4090 (24GB) |
| Qwen3-14B | 14.8B (14.8B) | Dense-GQA | 32.4 GB | 18.11 GB | 11.15 GB | NVIDIA RTX 3060 12GB (12GB) |
| Qwen3-8B | 8.2B (8.2B) | Dense-GQA | 18.67 GB | 10.76 GB | 6.9 GB | NVIDIA RTX 3060 12GB (12GB) |
| Qwen2.5-72B-Instruct | 72.7B (72.7B) | Dense-GQA | 152.93 GB | 82.73 GB | 48.56 GB | Apple M4 Pro Mac Mini (64GB) (64GB) |
| Qwen2.5-Coder-32B-Instruct | 32.8B (32.8B) | Dense-GQA | 70.23 GB | 38.55 GB | 23.14 GB | NVIDIA RTX 4090 (24GB) |
| Qwen2.5-14B-Instruct | 14.8B (14.8B) | Dense-GQA | 32.65 GB | 18.36 GB | 11.4 GB | NVIDIA RTX 3060 12GB (12GB) |
| Qwen2.5-7B-Instruct | 7.62B (7.62B) | Dense-GQA | 16.79 GB | 9.43 GB | 5.85 GB | NVIDIA RTX 3060 12GB (12GB) |
| QwQ-32B Reasoning | 32.8B (32.8B) | Dense-GQA | 70.23 GB | 38.55 GB | 23.14 GB | NVIDIA RTX 4090 (24GB) |
| Llama-4-Maverick-17B-128E (400B MoE) | 400B (17B) | MoE-GQA | 826.16 GB | 439.91 GB | 251.93 GB | 4x NVIDIA A100 80GB (320GB) (320GB) |
| Llama-4-Scout-17B-16E (109B MoE) | 109B (17B) | MoE-GQA | 226.7 GB | 121.45 GB | 70.22 GB | NVIDIA A100 80GB SXM (80GB) |
| Llama-3.3-70B-Instruct | 70.6B (70.6B) | Dense-GQA | 148.6 GB | 80.43 GB | 47.25 GB | NVIDIA RTX 6000 Ada (48GB) |
| Llama-3.1-405B-Instruct | 405B (405B) | Dense-GQA | 838.92 GB | 447.84 GB | 257.51 GB | 4x NVIDIA A100 80GB (320GB) (320GB) |
| Llama-3.1-70B-Instruct | 70.6B (70.6B) | Dense-GQA | 148.6 GB | 80.43 GB | 47.25 GB | NVIDIA RTX 6000 Ada (48GB) |
| Llama-3.1-8B-Instruct | 8.03B (8.03B) | Dense-GQA | 18.2 GB | 10.45 GB | 6.67 GB | NVIDIA RTX 3060 12GB (12GB) |
| Llama-3.2-3B-Instruct | 3.21B (3.21B) | Dense-GQA | 8.14 GB | 5.04 GB | 3.54 GB | NVIDIA RTX 3060 12GB (12GB) |
| Mistral-Large-2-123B | 123B (123B) | Dense-GQA | 256.8 GB | 138.03 GB | 80.23 GB | Apple M4 Max (128GB Unified) (128GB) |
| Mistral-Small-3.1-24B-Instruct | 24B (24B) | Dense-GQA | 51.35 GB | 28.17 GB | 16.9 GB | NVIDIA RTX 4090 (24GB) |
| Mixtral-8x22B-Instruct (MoE) | 141B (39B) | MoE-GQA | 292.87 GB | 156.72 GB | 90.46 GB | Apple M4 Max (128GB Unified) (128GB) |
| Mixtral-8x7B-Instruct (MoE) | 46.7B (12.9B) | MoE-GQA | 97.86 GB | 52.76 GB | 30.82 GB | NVIDIA RTX 5090 (32GB) |
| Codestral-22B | 22.2B (22.2B) | Dense-GQA | 48.14 GB | 26.71 GB | 16.27 GB | NVIDIA RTX 4090 (24GB) |
| Google Gemma-3-27B-IT | 27.4B (27.4B) | Dense-GQA | 60.98 GB | 34.52 GB | 21.64 GB | NVIDIA RTX 4090 (24GB) |
| Google Gemma-3-12B-IT | 12.2B (12.2B) | Dense-GQA | 28.79 GB | 17.01 GB | 11.28 GB | NVIDIA RTX 3060 12GB (12GB) |
| Google Gemma-3-4B-IT | 4.3B (4.3B) | Dense-GQA | 10.58 GB | 6.42 GB | 4.4 GB | NVIDIA RTX 3060 12GB (12GB) |
| Microsoft Phi-4 14B | 14.7B (14.7B) | Dense-GQA | 32.5 GB | 18.31 GB | 11.4 GB | NVIDIA RTX 3060 12GB (12GB) |
| Cohere Command-A 111B | 111B (111B) | Dense-GQA | 231.33 GB | 124.15 GB | 71.99 GB | NVIDIA A100 80GB SXM (80GB) |
| Yi-1.5-34B-Chat | 34.4B (34.4B) | Dense-GQA | 73.4 GB | 40.18 GB | 24.02 GB | NVIDIA RTX 5090 (32GB) |
| GLM-4-32B-0414 | 32B (32B) | Dense-GQA | 68.49 GB | 37.59 GB | 22.55 GB | NVIDIA RTX 4090 (24GB) |