How much VRAM do I need for a 70B model?
About 48.2 GB at Q4_K_M with 8k context (est.), based on Llama 3.3 70B Instruct. Two 24 GB cards (48 GB) are just short at 8k but fit at 4k, and IQ4_XS (about 42.6 GB) fits at 8k. A 64GB Mac gives the GPU about 48.0 GB by default, the same squeeze. One 24 GB card needs RAM offload and runs much slower.
Checked 2026-10-22 · memory figures are estimates
| Quant | Weights | Total needed (est., 8k ctx) |
|---|---|---|
| Q3_K_M | 35.3 GB | 40.3 GB |
| IQ4_XS | 37.5 GB | 42.6 GB |
| Q4_K_M | 42.8 GB | 48.2 GB |
| Q5_K_M | 50.2 GB | 55.9 GB |
| Q6_K | 57.9 GB | 63.5 GB |
| Q8_0 | 75.0 GB | 80.6 GB |
| F16 | 141.1 GB | 146.8 GB |
Which setups fit (Q4_K_M, 8k ctx, est.)
| Setup | Usable memory | Verdict |
|---|---|---|
| 2× RTX 3090 | 48.0 GB | Doesn't fit |
| 1× RTX 4090 + 64 GB RAM | 24.0 GB | Needs RAM offload |
| Apple M4 Max 64GB | ~48.0 GB | Doesn't fit |
How we got this number
Memory needed = parameters × bits per weight ÷ 8 (Q4_K_M is 4.85 bits, Q8_0 is 8.5), plus the KV cache for your context and runtime buffers. Worked example: Llama 3.3 70B Instruct has 70.55B parameters, so Q4_K_M weights are about 42.8 GB; with an 8k f16 KV cache (2.7 GB) and buffers the total is about 48.2 GB. All estimates. Full method.
Llama 3.3 70B Instruct page · 48 GB VRAM list · 64GB Mac list · Check 2× 3090 in the checker
FAQ
Can a 4090 run a 70B model?
Not fully. At Q4_K_M Llama 3.3 70B Instruct needs about 48.2 GB (est.) and a 4090 has 24 GB, so llama.cpp has to offload layers to system RAM. With 64 GB RAM the verdict is "Needs RAM offload" and it runs much slower than on GPU.
Q4 or Q8 for a 70B model?
Q8_0 needs about 80.6 GB against 48.2 GB for Q4_K_M (8k ctx, est.). Q8 is closer to full quality, but for most people with 48 GB the choice is Q4 or nothing.
How much does 32k context add for a 70B model?
For Llama 3.3 70B Instruct the f16 KV cache is about 10.7 GB at 32k against 1.3 GB at 4k (est.), so 32k adds about 9.4 GB. A q8_0 KV cache halves that.
Open the checker · What fits my GPU?
More short answers
- What LLM can I run on my GPU?
- Is 16GB RAM enough to run an LLM locally?
- MLX vs GGUF on a Mac: which is faster?
- How much VRAM do I need for a 7B model?
- How much VRAM do I need for a 32B model?
- Is 12GB VRAM enough for local LLMs?
- How much VRAM does context length use?
- Can you run an LLM on CPU only?
- Q4 vs Q8: which quantization should I use?
Will It Run? Pro NOT BUILT YET
A personal setup for your exact hardware (Ollama / llama.cpp configs ready for your coding agent), an alert when a new model fits, and a one-click cloud fallback when it doesn't.
Not built yet. Join the waitlist, $10/mo planned, no charge now. What's planned
Email only. No payment info.