Best local LLMs for 8 GB of VRAM
Short answer: the largest current models that fit fully in 8 GB are Gemma 4 12B (Q3_K_M), GLM 4.6V Flash (Q4_K_M), Qwen3.5 9B (Q4_K_M). 15 of 61 current models fit at Q3_K_M or better with an 8k context ESTIMATE.
Cards with 8 GB: RTX 5060 Ti 8GB, RTX 5060, RTX 5050, RTX 4060 Ti 8GB, RTX 4060, RTX 3070 Ti, RTX 3070, RTX 3060 Ti.
| Model | Params | Best quant that fits | Needs (8k ctx) | Headroom |
|---|---|---|---|---|
| Gemma 4 12B | 12.0B | Q3_K_M | 7.8 GB | 0.2 GB |
| GLM 4.6V Flash | 10.3B | Q4_K_M | 7.5 GB | 0.5 GB |
| Qwen3.5 9B | 9.7B | Q4_K_M | 7.0 GB | 1.0 GB |
| Ministral 3 8B (2512) | 8.9B | Q4_K_M | 7.4 GB | 0.6 GB |
| granite 4.2 8b | 8.8B | Q4_K_M | 7.5 GB | 0.5 GB |
| Qwen3 VL 8B | 8.8B | Q4_K_M | 7.4 GB | 0.6 GB |
| LFM2.5 8B A1B (MoE) MoE | 8.5B | Q6_K | 8.0 GB | 0.0 GB |
| Gemma 4 E4B | 8.0B | Q6_K | 7.6 GB | 0.4 GB |
| Gemma 4 E2B | 5.1B | Q8_0 | 6.4 GB | 1.6 GB |
| Qwen3.5 4B | 4.7B | Q8_0 | 6.1 GB | 1.9 GB |
| Qwen3 4B (2507) | 4.0B | Q8_0 | 6.3 GB | 1.7 GB |
| Nemotron Labs Dory 1.0 4B | 4.0B | Q8_0 | 5.5 GB | 2.5 GB |
| NVIDIA Nemotron 3 Nano 4B | 4.0B | Q8_0 | 5.2 GB | 2.8 GB |
| Ministral 3 3B (2512) | 3.8B | Q8_0 | 5.8 GB | 2.2 GB |
| Qwen3.5 2B | 2.3B | Q8_0 | 3.2 GB | 4.8 GB |
Best quant = highest of Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS, Q3_K_M that fits, f16 KV cache, 8k context. All memory figures are estimates (weights + KV cache + runtime buffers). Measured speeds per card are on the What fits my GPU? page and the checker.
FAQ
What is the biggest LLM that fits in 8 GB of VRAM?
By parameter count, Gemma 4 12B at Q3_K_M (about 7.8 GB at 8k context, estimate). Bigger models need RAM offload or more GPUs.
How many current open models fit in 8 GB?
15 of 61 current models in our dataset fit fully on the GPU at Q3_K_M or better with 8k context (estimate).
Other tiers: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 80 GB
Get the weekly "new models & what it takes to run them" email
Free. One email a week. Unsubscribe anytime.