Short answers

How much VRAM do I need for a 32B model?

On a 24 GB card by context (Q4_K_M, est.)

How we got this number

Memory needed = parameters × bits per weight ÷ 8 (Q4_K_M is 4.85 bits, Q8_0 is 8.5), plus the KV cache for your context and runtime buffers. Worked example: Qwen3 32B has 32.76B parameters, so Q4_K_M weights are about 19.9 GB; with an 8k f16 KV cache (2.1 GB) and buffers the total is about 23.6 GB. All estimates. Full method.

Qwen3 32B page · 24 GB VRAM list · Check an RTX 3090

FAQ

Can a 16 GB GPU run a 32B model?

Not fully at Q4_K_M, which needs about 23.6 GB (est.). It works with RAM offload at lower speed, or at Q3 with short context.

Q4 or Q5 for 32B on 24 GB?

Q5_K_M needs about 27.2 GB at 8k (est.), so on 24 GB, Q4_K_M is the practical choice if you want context room.

Does a MoE 30B need the same?

A 30B MoE like Qwen3 30B A3B needs similar memory to hold all experts, but runs faster per token. It can also offload experts to RAM with less slowdown.

Open the checker · What fits my GPU?

More short answers

• What LLM can I run on my GPU?
• How much VRAM do I need for a 70B model?
• Is 16GB RAM enough to run an LLM locally?
• MLX vs GGUF on a Mac: which is faster?
• How much VRAM do I need for a 7B model?
• Is 12GB VRAM enough for local LLMs?
• How much VRAM does context length use?
• Can you run an LLM on CPU only?
• Q4 vs Q8: which quantization should I use?