Best local LLMs for 12 GB of VRAM
Short answer: the largest current models that fit fully in 12 GB are gpt-oss 20B (MoE) (Q3_K_M), Phi 4 reasoning vision 15B (Q4_K_M), Ministral 3 14B (2512) (Q4_K_M). 18 of 61 current models fit at Q3_K_M or better with an 8k context ESTIMATE.
Cards with 12 GB: RTX 5070, RTX 4070 Ti, RTX 4070 Super, RTX 4070, RTX 3080 Ti, RTX 3080 12GB, RTX 3060 12GB, RTX 2060 12GB.
| Model | Params | Best quant that fits | Needs (8k ctx) | Headroom |
|---|---|---|---|---|
| gpt-oss 20B (MoE) MoE | 20.9B | Q3_K_M | 11.8 GB | 0.2 GB |
| Phi 4 reasoning vision 15B | 15.1B | Q4_K_M | 11.9 GB | 0.1 GB |
| Ministral 3 14B (2512) | 13.9B | Q4_K_M | 10.8 GB | 1.2 GB |
| Gemma 4 12B | 12.0B | Q6_K | 11.8 GB | 0.2 GB |
| GLM 4.6V Flash | 10.3B | Q6_K | 9.8 GB | 2.2 GB |
| Qwen3.5 9B | 9.7B | Q8_0 | 11.6 GB | 0.4 GB |
| Ministral 3 8B (2512) | 8.9B | Q8_0 | 11.7 GB | 0.3 GB |
| granite 4.2 8b | 8.8B | Q8_0 | 11.8 GB | 0.2 GB |
| Qwen3 VL 8B | 8.8B | Q8_0 | 11.6 GB | 0.4 GB |
| LFM2.5 8B A1B (MoE) MoE | 8.5B | Q8_0 | 10.1 GB | 1.9 GB |
| Gemma 4 E4B | 8.0B | Q8_0 | 9.7 GB | 2.3 GB |
| Gemma 4 E2B | 5.1B | Q8_0 | 6.4 GB | 5.6 GB |
| Qwen3.5 4B | 4.7B | Q8_0 | 6.1 GB | 5.9 GB |
| Qwen3 4B (2507) | 4.0B | Q8_0 | 6.3 GB | 5.7 GB |
| Nemotron Labs Dory 1.0 4B | 4.0B | Q8_0 | 5.5 GB | 6.5 GB |
| NVIDIA Nemotron 3 Nano 4B | 4.0B | Q8_0 | 5.2 GB | 6.8 GB |
| Ministral 3 3B (2512) | 3.8B | Q8_0 | 5.8 GB | 6.2 GB |
| Qwen3.5 2B | 2.3B | Q8_0 | 3.2 GB | 8.8 GB |
Best quant = highest of Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS, Q3_K_M that fits, f16 KV cache, 8k context. All memory figures are estimates (weights + KV cache + runtime buffers). Measured speeds per card are on the What fits my GPU? page and the checker.
FAQ
What is the biggest LLM that fits in 12 GB of VRAM?
By parameter count, gpt-oss 20B (MoE) at Q3_K_M (about 11.8 GB at 8k context, estimate). Bigger models need RAM offload or more GPUs.
How many current open models fit in 12 GB?
18 of 61 current models in our dataset fit fully on the GPU at Q3_K_M or better with 8k context (estimate).
Other tiers: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 80 GB
Get the weekly "new models & what it takes to run them" email
Free. One email a week. Unsubscribe anytime.