What LLM can I run on my GPU?
It depends on your VRAM. Largest dense model that fits at Q4_K_M with 8k context (est.): 8 GB about 10B, 12 GB about 15B, 16 GB about 15B, 24 GB about 33B, 48 GB about 33B. MoE models can go bigger with RAM offload. Pick your exact GPU in the What fits my GPU? picker to see every model.
Checked 2026-10-22 · memory figures are estimates
Largest model that fits, by VRAM (Q4_K_M, 8k ctx, est.)
| VRAM | Largest dense model | Needs | Largest overall | Needs | All models |
|---|---|---|---|---|---|
| 8 GB | GLM 4.6V Flash | 7.5 GB | GLM 4.6V Flash | 7.5 GB | 8 GB list |
| 12 GB | Phi 4 reasoning vision 15B | 11.9 GB | Phi 4 reasoning vision 15B | 11.9 GB | 12 GB list |
| 16 GB | Phi 4 reasoning vision 15B | 11.9 GB | gpt-oss 20B (MoE) | 14.1 GB | 16 GB list |
| 24 GB | DeepSeek R1 Distill Qwen 32B | 23.6 GB | Qwen3.6 35B A3B (MoE) | 23.7 GB | 24 GB list |
| 48 GB | DeepSeek R1 Distill Qwen 32B | 23.6 GB | Kimi Linear 48B A3B (MoE) | 32.1 GB | 48 GB list |
How we got this number
Memory needed = parameters × bits per weight ÷ 8 (Q4_K_M is 4.85 bits, Q8_0 is 8.5), plus the KV cache for your context and runtime buffers. Worked example: DeepSeek R1 Distill Qwen 32B has 32.76B parameters, so Q4_K_M weights are about 19.9 GB; with an 8k f16 KV cache (2.1 GB) and buffers the total is about 23.6 GB. All estimates. Full method.
FAQ
Can I run an LLM on a Mac mini?
Yes on Apple Silicon. The GPU uses unified memory, about 67% of it by default below 64 GB. Pick your memory size on our Mac page to see what fits.
What LLM can I run with 16GB VRAM?
At Q4_K_M with 8k context the largest dense model that fits is Phi 4 reasoning vision 15B (est.). See the 16 GB list for every model and the best quant for each.
Does a second GPU help?
For fit, yes. The checker sums VRAM across GPUs, so two 24 GB cards hold a model that needs up to about 48 GB. Speed is not additive.
Open the checker · What fits my GPU?
More short answers
- How much VRAM do I need for a 70B model?
- Is 16GB RAM enough to run an LLM locally?
- MLX vs GGUF on a Mac: which is faster?
- How much VRAM do I need for a 7B model?
- How much VRAM do I need for a 32B model?
- Is 12GB VRAM enough for local LLMs?
- How much VRAM does context length use?
- Can you run an LLM on CPU only?
- Q4 vs Q8: which quantization should I use?
Will It Run? Pro NOT BUILT YET
A personal setup for your exact hardware (Ollama / llama.cpp configs ready for your coding agent), an alert when a new model fits, and a one-click cloud fallback when it doesn't.
Not built yet. Join the waitlist, $10/mo planned, no charge now. What's planned
Email only. No payment info.