Which GPU for Which LLM — VRAM Requirements by Model (2026)

How to read the VRAM floor

Every floor in this guide comes from the same arithmetic rather than from a vendor chart: Q4_K_M weights run roughly 0.55 GB per billion parameters, and the runtime plus a usable context window wants about 2 GB on top. A 14B model is therefore ~9.7 GB resident, a 32B ~19.6 GB, and a 70B ~40 GB before context. Round up to the nearest card capacity and that is the floor.

The figure that stops being predictable is what happens below the floor. Once weights spill past VRAM the number is set by PCIe and system-memory bandwidth, not by the GPU, and the drop is steep rather than gradual — which is why a tier is quoted as a floor and not as a suggestion. Per-card medians over published runs, with the run counts and sources behind them, live on the benchmark pages; the GPU picker ranks the cards that clear a floor and fit your budget by median measured tok/s.

Qwen3 — VRAM floors and GPU picks

Variants 0.6B · 1.7B · 4B · 8B · 14B · 32B · 30B-A3B MoE · 235B-A22B MoE · quantisation Q4_K_M · context 128K · full Qwen3 build guide →

Perfect for chatbot + coding use cases on smaller variants. 16GB lets you run 14B at Q4 with comfortable context.

Llama 4 — VRAM floors and GPU picks

Variants Scout 109B MoE (active 17B) · Maverick 400B MoE (active 17B) · quantisation Q4_K_M · context 10M (Scout) · 1M (Maverick) · full Llama 4 build guide →

Scout's ~60 GB of Q4 weights do not fit a 24 GB card, but only 17B parameters are active per token, so keeping attention and shared layers on the GPU and the routed experts in system RAM stays usable for chat. A ~2-bit quant (~40 GB) cuts the RAM need at a quality cost.

DeepSeek V3.2 — VRAM floors and GPU picks

Variants 671B MoE (active ~37B) · quantisation Q4_K_M · context 128K · full DeepSeek V3.2 build guide →

A ~2-bit dynamic quant shrinks the weights to roughly 170–230 GB. The 24 GB card holds attention and the shared expert; the routed experts live in system RAM (llama.cpp tensor overrides or ktransformers). Workable for batch jobs, slow for chat, and ~2-bit loses noticeably more quality than Q4.

Gemma 4 — VRAM floors and GPU picks

Variants 2B · 7B · 9B · 27B · quantisation Q4_K_M · context 128K · full Gemma 4 build guide →

Gemma 4 2B at Q4 fits even on integrated graphics. 7B Q4 needs 8 GB; 12 GB lets you keep full 128K context active.

Phi-4 — VRAM floors and GPU picks

Variants mini (4B) · 14B · quantisation Q4_K_M · context 16K · full Phi-4 build guide →

Phi-4 mini fits on almost any modern card. Quality is solid for a 4B model — beats Llama 3.2 3B on most benchmarks.

Frequently asked questions

Where to go next

• Will it run? The model × GPU matrix — one page per exact pair, with cited speeds and a free CSV.
• Rank GPUs by VRAM floor, budget and workload — the same mapping, indexed by what you can spend.
• The local-LLM model-fit table — every card SpecPicks holds runs for, bucketed by what it loads at Q4.
• Best GPUs for local LLMs — the ranked buying guide, per VRAM bracket.
• Benchmark data by card — medians, run counts and source links per SKU.