Can you run an LLM on CPU only?
Yes. llama.cpp runs models on the CPU from system RAM, and speed depends mostly on memory bandwidth. With 32 GB RAM about 26 GB is free after the OS (est.), enough for Qwen3.6 35B A3B (MoE) at Q4_K_M. MoE models with few active parameters run much faster on CPU. We have no measured CPU-only speeds yet.
Checked 2026-10-22 · memory figures are estimates
Models that fit in 32 GB RAM (Q4_K_M, 4k ctx, est.)
| Model | Needs |
|---|---|
| Qwen3.6 35B A3B (MoE) MoE | 23.6 GB |
| Qwen3.5 35B A3B (MoE) MoE | 23.6 GB |
| Olmo 3.1 32B | 22.2 GB |
| NVIDIA Nemotron 3.5 Lightning 30B A3B (MoE) MoE | 20.7 GB |
| NVIDIA Nemotron 3 Nano 30B A3B (MoE) MoE | 20.7 GB |
| Gemma 4 31B | 22.0 GB |
| GLM 4.7 Flash MoE | 20.7 GB |
| Qwen3 Coder 30B A3B (MoE) MoE | 20.4 GB |
| Qwen3 30B A3B (2507) (MoE) MoE | 20.4 GB |
| granite 4.2 30b | 20.3 GB |
| Raon-1 mini preview 28.8B A4.3B (MoE) MoE | 19.0 GB |
| Qwen3.8 27B | 18.6 GB |
How we got this number
Free RAM = total minus about 6 GB for the OS. Memory needed uses the same formula as GPUs (method). Speed: each generated token reads the active weights from RAM, so a dense model is limited by memory bandwidth and an MoE model reads only its active experts. We don't publish CPU speed estimates; submit a llama-bench CPU run and we'll add it.
FAQ
Can I run DeepSeek locally without a GPU?
The big DeepSeek models need hundreds of GB even at Q4, so CPU-only means a server with that much RAM. Smaller distills fit ordinary PCs. Check sizes on each model page.
Is DDR5 faster than DDR4 for CPU inference?
Usually, because generation speed is bound by memory bandwidth and DDR5 has more. We haven't measured it ourselves.
Should I use a dense or MoE model on CPU?
MoE. A model like Qwen3.6 35B A3B activates about 3B parameters per token, so it reads far less memory per token than a dense 27B.
Open the checker · What fits my GPU?
More short answers
- What LLM can I run on my GPU?
- How much VRAM do I need for a 70B model?
- Is 16GB RAM enough to run an LLM locally?
- MLX vs GGUF on a Mac: which is faster?
- How much VRAM do I need for a 7B model?
- How much VRAM do I need for a 32B model?
- Is 12GB VRAM enough for local LLMs?
- How much VRAM does context length use?
- Q4 vs Q8: which quantization should I use?
Will It Run? Pro NOT BUILT YET
A personal setup for your exact hardware (Ollama / llama.cpp configs ready for your coding agent), an alert when a new model fits, and a one-click cloud fallback when it doesn't.
Not built yet. Join the waitlist, $10/mo planned, no charge now. What's planned
Email only. No payment info.