Best local LLMs for 16 GB of VRAM
Short answer: the largest current models that fit fully in 16 GB are Qwen3.8 27B (Q3_K_M), Qwen3.6 27B (Q3_K_M), Qwen3.5 27B (Q3_K_M). 24 of 61 current models fit at Q3_K_M or better with an 8k context ESTIMATE.
Cards with 16 GB: RTX 5080, RTX 5070 Ti, RTX 5060 Ti 16GB, RTX 4080 Super, RTX 4080, RTX 4070 Ti Super, RTX 4060 Ti 16GB, RTX 4090 Laptop.
| Model | Params | Best quant that fits | Needs (8k ctx) | Headroom |
|---|---|---|---|---|
| Qwen3.8 27B | 27.8B | Q3_K_M | 15.7 GB | 0.3 GB |
| Qwen3.6 27B | 27.8B | Q3_K_M | 15.7 GB | 0.3 GB |
| Qwen3.5 27B | 27.8B | Q3_K_M | 15.7 GB | 0.3 GB |
| Gemma 4 26B A4B (MoE) MoE | 25.8B | IQ4_XS | 15.5 GB | 0.5 GB |
| Devstral Small 2 24B (2512) | 24.0B | IQ4_XS | 15.3 GB | 0.7 GB |
| LFM2.5 24B A2B (MoE) MoE | 24.0B | IQ4_XS | 14.1 GB | 1.9 GB |
| gpt-oss 20B (MoE) MoE | 20.9B | Q4_K_M | 14.1 GB | 1.9 GB |
| Phi 4 reasoning vision 15B | 15.1B | Q6_K | 15.3 GB | 0.7 GB |
| Ministral 3 14B (2512) | 13.9B | Q6_K | 13.9 GB | 2.1 GB |
| Gemma 4 12B | 12.0B | Q8_0 | 14.8 GB | 1.2 GB |
| GLM 4.6V Flash | 10.3B | Q8_0 | 12.4 GB | 3.6 GB |
| Qwen3.5 9B | 9.7B | Q8_0 | 11.6 GB | 4.4 GB |
| Ministral 3 8B (2512) | 8.9B | Q8_0 | 11.7 GB | 4.3 GB |
| granite 4.2 8b | 8.8B | Q8_0 | 11.8 GB | 4.2 GB |
| Qwen3 VL 8B | 8.8B | Q8_0 | 11.6 GB | 4.4 GB |
| LFM2.5 8B A1B (MoE) MoE | 8.5B | Q8_0 | 10.1 GB | 5.9 GB |
| Gemma 4 E4B | 8.0B | Q8_0 | 9.7 GB | 6.3 GB |
| Gemma 4 E2B | 5.1B | Q8_0 | 6.4 GB | 9.6 GB |
| Qwen3.5 4B | 4.7B | Q8_0 | 6.1 GB | 9.9 GB |
| Qwen3 4B (2507) | 4.0B | Q8_0 | 6.3 GB | 9.7 GB |
| Nemotron Labs Dory 1.0 4B | 4.0B | Q8_0 | 5.5 GB | 10.5 GB |
| NVIDIA Nemotron 3 Nano 4B | 4.0B | Q8_0 | 5.2 GB | 10.8 GB |
| Ministral 3 3B (2512) | 3.8B | Q8_0 | 5.8 GB | 10.2 GB |
| Qwen3.5 2B | 2.3B | Q8_0 | 3.2 GB | 12.8 GB |
Best quant = highest of Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS, Q3_K_M that fits, f16 KV cache, 8k context. All memory figures are estimates (weights + KV cache + runtime buffers). Measured speeds per card are on the What fits my GPU? page and the checker.
FAQ
What is the biggest LLM that fits in 16 GB of VRAM?
By parameter count, Qwen3.8 27B at Q3_K_M (about 15.7 GB at 8k context, estimate). Bigger models need RAM offload or more GPUs.
How many current open models fit in 16 GB?
24 of 61 current models in our dataset fit fully on the GPU at Q3_K_M or better with 8k context (estimate).
Other tiers: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 80 GB
Get the weekly "new models & what it takes to run them" email
Free. One email a week. Unsubscribe anytime.