Best local LLMs for 48 GB of VRAM
Short answer: the largest current models that fit fully in 48 GB are Qwen3 Coder Next (MoE) (IQ4_XS), NVIDIA Nemotron 3.5 Bolt 74B A9B (MoE) (IQ4_XS), Kimi Linear 48B A3B (MoE) (Q6_K). 37 of 61 current models fit at Q3_K_M or better with an 8k context ESTIMATE.
Cards with 48 GB: RTX PRO 5000 Blackwell 48GB, RTX 6000 Ada, RTX A6000.
| Model | Params | Best quant that fits | Needs (8k ctx) | Headroom |
|---|---|---|---|---|
| Qwen3 Coder Next (MoE) MoE | 79.7B | IQ4_XS | 45.2 GB | 2.8 GB |
| NVIDIA Nemotron 3.5 Bolt 74B A9B (MoE) MoE | 74.9B | IQ4_XS | 42.4 GB | 5.6 GB |
| Kimi Linear 48B A3B (MoE) MoE | 49.1B | Q6_K | 43.1 GB | 4.9 GB |
| Qwen3.6 35B A3B (MoE) MoE | 36.0B | Q8_0 | 40.9 GB | 7.1 GB |
| Qwen3.5 35B A3B (MoE) MoE | 36.0B | Q8_0 | 40.9 GB | 7.1 GB |
| Olmo 3.1 32B | 32.2B | Q8_0 | 37.9 GB | 10.1 GB |
| NVIDIA Nemotron 3.5 Lightning 30B A3B (MoE) MoE | 31.6B | Q8_0 | 35.9 GB | 12.1 GB |
| NVIDIA Nemotron 3 Nano 30B A3B (MoE) MoE | 31.6B | Q8_0 | 35.9 GB | 12.1 GB |
| Gemma 4 31B | 31.3B | Q8_0 | 37.7 GB | 10.3 GB |
| GLM 4.7 Flash MoE | 31.2B | Q8_0 | 35.9 GB | 12.1 GB |
| Qwen3 Coder 30B A3B (MoE) MoE | 30.5B | Q8_0 | 35.5 GB | 12.5 GB |
| Qwen3 30B A3B (2507) (MoE) MoE | 30.5B | Q8_0 | 35.5 GB | 12.5 GB |
| granite 4.2 30b | 29.3B | Q8_0 | 35.4 GB | 12.6 GB |
| Qwen3.8 27B | 27.8B | Q8_0 | 32.1 GB | 15.9 GB |
| Qwen3.6 27B | 27.8B | Q8_0 | 32.1 GB | 15.9 GB |
| Qwen3.5 27B | 27.8B | Q8_0 | 32.1 GB | 15.9 GB |
| Gemma 4 26B A4B (MoE) MoE | 25.8B | Q8_0 | 29.9 GB | 18.1 GB |
| Devstral Small 2 24B (2512) | 24.0B | Q8_0 | 28.7 GB | 19.3 GB |
| LFM2.5 24B A2B (MoE) MoE | 24.0B | Q8_0 | 27.5 GB | 20.5 GB |
| gpt-oss 20B (MoE) MoE | 20.9B | Q8_0 | 24.1 GB | 23.9 GB |
| Phi 4 reasoning vision 15B | 15.1B | Q8_0 | 19.1 GB | 28.9 GB |
| Ministral 3 14B (2512) | 13.9B | Q8_0 | 17.5 GB | 30.5 GB |
| Gemma 4 12B | 12.0B | Q8_0 | 14.8 GB | 33.2 GB |
| GLM 4.6V Flash | 10.3B | Q8_0 | 12.4 GB | 35.6 GB |
| Qwen3.5 9B | 9.7B | Q8_0 | 11.6 GB | 36.4 GB |
| Ministral 3 8B (2512) | 8.9B | Q8_0 | 11.7 GB | 36.3 GB |
| granite 4.2 8b | 8.8B | Q8_0 | 11.8 GB | 36.2 GB |
| Qwen3 VL 8B | 8.8B | Q8_0 | 11.6 GB | 36.4 GB |
| LFM2.5 8B A1B (MoE) MoE | 8.5B | Q8_0 | 10.1 GB | 37.9 GB |
| Gemma 4 E4B | 8.0B | Q8_0 | 9.7 GB | 38.3 GB |
| Gemma 4 E2B | 5.1B | Q8_0 | 6.4 GB | 41.6 GB |
| Qwen3.5 4B | 4.7B | Q8_0 | 6.1 GB | 41.9 GB |
| Qwen3 4B (2507) | 4.0B | Q8_0 | 6.3 GB | 41.7 GB |
| Nemotron Labs Dory 1.0 4B | 4.0B | Q8_0 | 5.5 GB | 42.5 GB |
| NVIDIA Nemotron 3 Nano 4B | 4.0B | Q8_0 | 5.2 GB | 42.8 GB |
| Ministral 3 3B (2512) | 3.8B | Q8_0 | 5.8 GB | 42.2 GB |
| Qwen3.5 2B | 2.3B | Q8_0 | 3.2 GB | 44.8 GB |
Best quant = highest of Q8_0, Q6_K, Q5_K_M, Q4_K_M, IQ4_XS, Q3_K_M that fits, f16 KV cache, 8k context. All memory figures are estimates (weights + KV cache + runtime buffers). Measured speeds per card are on the What fits my GPU? page and the checker.
FAQ
What is the biggest LLM that fits in 48 GB of VRAM?
By parameter count, Qwen3 Coder Next (MoE) at IQ4_XS (about 45.2 GB at 8k context, estimate). Bigger models need RAM offload or more GPUs.
How many current open models fit in 48 GB?
37 of 61 current models in our dataset fit fully on the GPU at Q3_K_M or better with 8k context (estimate).
Other tiers: 8 GB · 12 GB · 16 GB · 24 GB · 32 GB · 48 GB · 80 GB
Get the weekly "new models & what it takes to run them" email
Free. One email a week. Unsubscribe anytime.