Gemma 4 spans phone-class E2B/E4B models up to a 31B dense flagship. The 12B is the "fits in 12 GB VRAM" daily driver; the 26B-A4B mixture-of-experts activates only about 4B parameters per token, so on a 24 GB card it runs several times faster than the dense 31B.
Hardware tiers for running Gemma 4
🧪
Minimum (E2B / E4B at Q4)
E2B Q4_K_M weights are about 3.1 GB and E4B about 5.0 GB, so both fit an 8 GB card with room for context; E2B also runs on a 4 GB card.
VRAM8 GB
System RAM16 GB DDR5
ThroughputNo cited Gemma 4 run on an 8 GB card yet · other cards: ~40 tok/s on E2B Q4_0 (4 GB GTX 1650, 1 cited run) · ~88 tok/s on E4B (12 GB RTX 4070 Ti, a faster card; XDA Developers)
*Price sourced from Amazon.com. Price and availability subject to change.
🏆
Recommended (12B at Q4 or Q8)
12B is the daily-driver sweet spot: a 12 GB card holds it at Q4, and 16 GB holds it at Q4 with long context or at Q8. 26B-A4B and 31B do not fit here: their Q4_K_M weights are about 17.0 GB and 18.3 GB before context, so a 16 GB card needs a Q3 quant or CPU offload.
VRAM16 GB
System RAM32 GB DDR5
Throughput~39 tok/s on 12B Q4 on a 16 GB RX 9060 XT (llama.cpp ROCm discussion) · ~65 tok/s on a 16 GB RX 9070 XT (LocalAIMaster) · ~28 tok/s on a 12 GB RTX 3060 (llmrun.dev) · no cited run yet on the cards below
*Price sourced from Amazon.com. Price and availability subject to change.
⚡
Luxury (31B or 26B-A4B at Q4)
A 24 GB card (RTX 3090, RTX 4090) holds Gemma 4 31B at Q4 (about 18.3 GB of weights) with room for context. 31B at Q8 is about 32.6 GB, more than even the 32 GB RTX 5090 holds, so Q8 needs two cards or offload; 26B-A4B at Q8 (about 26.9 GB) fits a 5090.
VRAM24 GB
System RAM64 GB DDR5
Throughput~35 tok/s on 31B Q4 (RTX 3090) · ~61 tok/s on 31B Q4 (RTX 5090) · ~119 tok/s on 26B-A4B (RTX 3090), all Hardware Corner
Match it to VRAM: E2B/E4B for 4-8 GB cards, 12B for 12-16 GB, and 31B or the 26B-A4B MoE for 24 GB. Pick 26B-A4B over 31B when speed matters: it activates only about 4B parameters per token, so it generates several times faster on the same card.
Is Gemma 4 multimodal?
Yes. The E4B, 12B and 31B checkpoints all ship a vision encoder in their Hugging Face configs, and the GGUF repos include the mmproj projector file llama.cpp needs for image input.
What's the cheapest GPU for Gemma 4 31B?
A used RTX 3090. Its 24 GB holds 31B at Q4 with context, and Hardware Corner measured about 35 tok/s on it. An RTX 5090 is faster (about 61 tok/s), but the 3090 wins on price per token.
More guides & deep dives from the SpecPicks archive
