Models › gpt-oss 20B (MoE) › RTX 4090
Can I run gpt-oss 20B (MoE) on RTX 4090?
Yes
Yes. gpt-oss 20B (MoE) fits fully on the RTX 4090 at Q6_K (about 18.8 GB of 24 GB usable, 8k context). ESTIMATE
Verdict by quantization (8k context, f16 KV, 32 GB RAM for offload)
| Quant | Needs | RTX 4090 (24 GB) |
|---|---|---|
| Q3_K_M | 11.8 GB | ✅ Fits |
| IQ4_XS | 12.5 GB | ✅ Fits |
| Q4_K_M | 14.1 GB | ✅ Fits |
| MXFP4 | 13.5 GB | ✅ Fits |
| Q5_K_M | 16.4 GB | ✅ Fits |
| Q6_K | 18.8 GB | ✅ Fits |
| Q8_0 | 24.1 GB | 🐢 Needs RAM offload |
| F16 | 44.7 GB | 🐢 Needs RAM offload |
Measured speed MEASURED
| Quant | Generation (tok/s) | Prompt (tok/s) | Source |
|---|---|---|---|
| MXFP4 | 221.95 | — | llama.cpp discussion (2026-10-20) |
| MXFP4 | 225.22 | — | llama.cpp discussion (2026-10-20) |
| MXFP4 | 271.99 | — | llama.cpp discussion (2026-10-20) |
| MXFP4 | 232.05 | — | llama.cpp discussion (2026-10-20) |
Memory figures are estimates from the model's parameter count and config.json (Hugging Face). How we calculate. Last verified with data from 2026-10-20.
FAQ
Can I run gpt-oss 20B (MoE) on the RTX 4090?
Yes. gpt-oss 20B (MoE) fits fully on the RTX 4090 at Q6_K (about 18.8 GB of 24 GB usable, 8k context).
How much VRAM does gpt-oss 20B (MoE) need?
About 14.1 GB at Q4_K_M and 24.1 GB at Q8_0 with 8k context (estimate).
gpt-oss 20B (MoE) on other hardware: RTX 3060 12GB · RTX 4060 Ti 16GB · RTX 3090 · RTX 5090 · Apple M4 Pro (48 GB) · Apple M4 Max (128 GB)
Other models on RTX 4090: Llama 3.1 8B Instruct
See all models for 24 GB VRAM.
Won't fit? Free hosted options: FreeAISwap.