Best local LLMs for 80 GB of VRAM
Short answer: the largest current models that fit fully in 80 GB are Mistral Medium 3.5 128B (IQ4_XS), NVIDIA Nemotron 3.5 Super VL 120B A12B (MoE) (Q4_K_M), NVIDIA Nemotron 3 Super 120B A12B (MoE) (Q4_K_M). 44 of 61 current models fit at Q3_K_M or better with an 8k context ESTIMATE.
No single consumer card in our list has exactly 80 GB; this tier covers datacenter cards or multi-GPU totals (the checker sums VRAM across GPUs).
| Model | Params | Best quant that fits | Needs (8k ctx) | Headroom |
|---|---|---|---|---|
| Mistral Medium 3.5 128B | 127.7B | IQ4_XS | 73.8 GB | 6.2 GB |
| NVIDIA Nemotron 3.5 Super VL 120B A12B (MoE) MoE | 124.5B | Q4_K_M | 78.5 GB | 1.5 GB |
| NVIDIA Nemotron 3 Super 120B A12B (MoE) MoE | 123.6B | Q4_K_M | 78.0 GB | 2.0 GB |
| Mistral Small 4 119B (2603) (MoE) MoE | 119.4B | Q4_K_M | 75.6 GB | 4.4 GB |
| gpt oss 120b (MoE) MoE | 116.8B | Q4_K_M | 74.1 GB | 5.9 GB |
| GLM 4.5 Air (MoE) MoE | 110.5B | Q4_K_M | 71.5 GB | 8.5 GB |
| Llama 4 Scout 17B 16E (MoE) MoE | 108.6B | Q4_K_M | 70.5 GB | 9.5 GB |
| Qwen3 Coder Next (MoE) MoE | 79.7B | Q6_K | 68.5 GB | 11.5 GB |
| NVIDIA Nemotron 3.5 Bolt 74B A9B (MoE) MoE | 74.9B | Q6_K | 64.5 GB | 15.5 GB |
| Kimi Linear 48B A3B (MoE) MoE | 49.1B | Q8_0 | 55.4 GB | 24.6 GB |
| Qwen3.6 35B A3B (MoE) MoE | 36.0B | Q8_0 | 40.9 GB | 39.1 GB |
| Qwen3.5 35B A3B (MoE) MoE | 36.0B | Q8_0 | 40.9 GB | 39.1 GB |
| Olmo 3.1 32B | 32.2B | Q8_0 | 37.9 GB | 42.1 GB |
| NVIDIA Nemotron 3.5 Lightning 30B A3B (MoE) MoE | 31.6B | Q8_0 | 35.9 GB | 44.1 GB |
| NVIDIA Nemotron 3 Nano 30B A3B (MoE) MoE | 31.6B | Q8_0 | 35.9 GB | 44.1 GB |
| Gemma 4 31B | 31.3B | Q8_0 | 37.7 GB | 42.3 GB |
| GLM 4.7 Flash MoE | 31.2B | Q8_0 | 35.9 GB | 44.1 GB |
| Qwen3 Coder 30B A3B (MoE) MoE | 30.5B | Q8_0 | 35.5 GB | 44.5 GB |
| Qwen3 30B A3B (2507) (MoE) MoE | 30.5B | Q8_0 | 35.5 GB | 44.5 GB |
| granite 4.2 30b | 29.3B | Q8_0 | 35.4 GB | 44.6 GB |
| Qwen3.8 27B | 27.8B | Q8_0 | 32.1 GB | 47.9 GB |
| Qwen3.6 27B | 27.8B | Q8_0 | 32.1 GB | 47.9 GB |
| Qwen3.5 27B | 27.8B | Q8_0 | 32.1 GB | 47.9 GB |
| Gemma 4 26B A4B (MoE) MoE | 25.8B | Q8_0 | 29.9 GB | 50.1 GB |
| Devstral Small 2 24B (2512) | 24.0B | Q8_0 | 28.7 GB | 51.3 GB |
| LFM2.5 24B A2B (MoE) MoE | 24.0B | Q8_0 | 27.5 GB | 52.5 GB |
| gpt-oss 20B (MoE) MoE | 20.9B | Q8_0 | 24.1 GB | 55.9 GB |
| Phi 4 reasoning vision 15B | 15.1B | Q8_0 | 19.1 GB | 60.9 GB |
| Ministral 3 14B (2512) | 13.9B | Q8_0 | 17.5 GB | 62.5 GB |
| Gemma 4 12B | 12.0B | Q8_0 | 14.8 GB | 65.2 GB |
| GLM 4.6V Flash | 10.3B | Q8_0 | 12.4 GB | 67.6 GB |
| Qwen3.5 9B | 9.7B | Q8_0 | 11.6 GB | 68.4 GB |
| Ministral 3 8B (2512) | 8.9B | Q8_0 | 11.7 GB | 68.3 GB |
| granite 4.2 8b | 8.8B | Q8_0 | 11.8 GB | 68.2 GB |
| Qwen3 VL 8B | 8.8B | Q8_0 | 11.6 GB | 68.4 GB |
| LFM2.5 8B A1B (MoE) MoE | 8.5B | Q8_0 | 10.1 GB | 69.9 GB |
| Gemma 4 E4B | 8.0B | Q8_0 | 9.7 GB | 70.3 GB |
| Gemma 4 E2B | 5.1B | Q8_0 | 6.4 GB | 73.6 GB |
| Qwen3.5 4B | 4.7B | Q8_0 | 6.1 GB | 73.9 GB |
| Qwen3 4B (2507) | 4.0B | Q8_0 | 6.3 GB | 73.7 GB |
| Nemotron Labs Dory 1.0 4B | 4.0B | Q8_0 | 5.5 GB | 74.5 GB |
| NVIDIA Nemotron 3 Nano 4B | 4.0B | Q8_0 | 5.2 GB | 74.8 GB |
| Ministral 3 3B (2512) | 3.8B | Q8_0 | 5.8 GB | 74.2 GB |
| Qwen3.5 2B | 2.3B | Q8_0 | 3.2 GB | 76.8 GB |
Best quant = highest of Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS, Q3_K_M that fits, f16 KV cache, 8k context. All memory figures are estimates (weights + KV cache + runtime buffers). Measured speeds per card are on the What fits my GPU? page and the checker.
FAQ
What is the biggest LLM that fits in 80 GB of VRAM?
By parameter count, Mistral Medium 3.5 128B at IQ4_XS (about 73.8 GB at 8k context, estimate). Bigger models need RAM offload or more GPUs.
How many current open models fit in 80 GB?
44 of 61 current models in our dataset fit fully on the GPU at Q3_K_M or better with 8k context (estimate).
Other tiers: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 80 GB
Get the weekly "new models & what it takes to run them" email
Free. One email a week. Unsubscribe anytime.