gpt-oss 20B vs Phi-4 14B on a 12 GB GPU: Which One Actually Fits?

Quick Answer

A 12 GB card runs both, but only one comfortably. gpt-oss 20B ships natively in MXFP4 and OpenAI states it "runs within 16GB of memory" (OpenAI) — on a 12 GB RTX 3060 that means partial offload once context grows. Phi-4 14B at Q4_K_M stays fully resident with room for an 8K KV cache. For a single 12 GB card, Phi-4 is the safer default.

Anyone shopping for a local assistant in 2026 runs into the same wall twice. The first time is when the model card says 20B and the card says 12 GB, and the arithmetic does not obviously fail — MXFP4 weights are small enough that the answer looks like "maybe." The second time is three days later, when context has grown past a few thousand tokens, the KV cache has eaten the margin, and generation has quietly dropped by more than half because layers are now being served from system RAM.

This piece is for the builder with exactly one 12 GB card — most often an MSI GeForce RTX 3060 Ventus 3X 12G OC bought used — who wants a 14-to-20B-class assistant that stays resident in VRAM and answers fast. It is not a 70B fantasy piece. The question is narrow and answerable: between gpt-oss 20B and Phi-4 14B, which one actually fits, and what does the loser cost you?

The short version is that these two models fail differently. gpt-oss 20B is a mixture-of-experts model whose released weights are natively quantized, which is why it gets closer to fitting than a 20B dense model ever would. Phi-4 is a 14B dense model that was never going to be tight on 12 GB in the first place. The interesting comparison is not raw capability but what each one leaves you after the weights are loaded — because that leftover is your context window, and your context window is what you actually use.

Everything below is synthesis of published model cards, vendor specifications, and community-reported measurements. Figures are cited inline. Where a number varies too much across runtimes and quantization builds to state honestly, it is given as a range.

Key Takeaways

• gpt-oss 20B fits, barely. OpenAI targets 16 GB of memory for the 20B checkpoint (OpenAI); a 12 GB board holds the weights but leaves little for KV cache.
• Phi-4 14B fits with room. A 14B dense model at Q4_K_M lands near 9 GB, leaving roughly 2-3 GB of a 12 GB frame buffer for cache and runtime overhead.
• The quant floor differs. gpt-oss 20B is already at its native MXFP4 precision — there is no free step down. Phi-4 has Q4_K_M, Q5_K_M, and Q6_K rungs to trade against.
• Context ceiling decides the winner. Both models advertise long context; on 12 GB, KV cache growth, not parameter count, is what pushes you off the GPU.
• Offload is a bandwidth cliff, not a slope. The RTX 3060's 360 GB/s of GDDR6 (TechPowerUp) versus roughly 51 GB/s from dual-channel DDR4-3200 is a ~7× gap that a fast CPU cannot close.

How much VRAM does each model actually need?

The two models are structurally different, which is why a straight parameter-count comparison misleads. gpt-oss 20B is a mixture-of-experts design with roughly 21 billion total parameters but only a fraction active per token, released with MXFP4 quantization applied to the expert weights rather than bolted on afterward (OpenAI, model card). Phi-4 is a conventional 14-billion-parameter dense transformer, documented in Microsoft's technical report (arXiv 2412.08905), and quantized after the fact by the community into the usual GGUF ladder.

Read that table as a fit test against a 12,288 MB frame buffer minus roughly 400-700 MB of driver, display, and runtime overhead. The honest reading is that Phi-4 at Q4_K_M is the only row with real headroom, Q5_K_M is tight but workable on a headless box, and both Q6_K and gpt-oss 20B are over the line on a 12 GB card.

Does gpt-oss 20B fit in 12 GB?

The argument for gpt-oss 20B on small cards is native quantization. Most quantized models you download are a lossy conversion of bf16 weights done by a third party; gpt-oss shipped with MXFP4 applied to the MoE layers as part of the release, which is why OpenAI's own guidance targets a 16 GB memory envelope rather than the 40 GB-plus a 21B dense bf16 model would demand (OpenAI). That is a genuinely different starting position, and it is why the model shows up in 12 GB conversations at all.

The problem is that 16 GB is not 12 GB. With weights consuming most of the frame buffer, what remains for the KV cache is small, and the KV cache is not optional — it is where the conversation lives. At short prompts and short replies you can stay on the card. As soon as you paste a long document, load a system prompt with tool definitions, or run a multi-turn session, the cache grows and the runtime starts placing layers in system memory.

The practical mode on a 12 GB board is therefore partial offload from the start: keep as many layers on the GPU as fit, accept that the rest are served across PCIe. Runtimes in the llama.cpp family expose this directly as a layer count (llama.cpp), and tuning it is the difference between a model that is slow and a model that is unusable. It works. It is not the experience of a resident model.

Does Phi-4 14B fit in 12 GB?

Yes, and this is the unglamorous part of the answer. Phi-4 is 14B dense, documented at 14 billion parameters with a 16K-token context window in Microsoft's technical report (arXiv 2412.08905). At Q4_K_M — llama.cpp's default recommendation for a quality-to-size balance (llama.cpp) — the GGUF lands near 9 GB, which leaves roughly 2.5-3 GB after driver overhead. That is enough for a genuinely useful context window with the whole model resident.

Stepping up the ladder costs that headroom quickly. Q5_K_M adds roughly 1.5 GB and is still loadable on a headless 12 GB card, but the cache margin shrinks to the point where long sessions start spilling. Q6_K effectively consumes the card. Q8_0 and fp16 are not 12 GB propositions for a 14B model at all.

The quality question is the one people over-index on. Community perplexity comparisons consistently show the gap between Q4_K_M and Q5_K_M as small and the gap between Q4_K_M and Q3-and-below as large — the ladder is not linear, and Q4_K_M sits at the knee (llama.cpp). Buying a quality step up from Q4_K_M to Q5_K_M by giving up your context window is, on this card, a bad trade.

Quantization matrix

The figures below are for a 14B dense model on a 12 GB card and reflect the range the community reports across runtimes and driver versions rather than a single measured configuration. Treat the tok/s column as an order-of-magnitude guide, not a benchmark result.

Two rows matter. Q4_K_M is the row you want, and Q3_K_M is the row that signals you have outgrown the card — if you are quantizing below Q4 to make something fit, the correct fix is a smaller model or a bigger card, not a smaller quant.

Which is faster in practice?

Throughput on a 12 GB card is dominated by one variable: whether the model is resident. Below, "resident" means all layers on the GPU and "offload" means some layers served from system RAM. The RTX 3060's memory subsystem — 12 GB of GDDR6 on a 192-bit bus, 360 GB/s of bandwidth, 170 W TGP (TechPowerUp) — is what the resident numbers are made of.

The pattern generalizes past any specific tok/s figure: a resident 14B model at 8K context on this class of card sits in the low-tens of tokens per second, and every layer you push to system RAM subtracts from that non-linearly. Prefill — processing the prompt you just pasted — is compute-bound and degrades more gracefully than generation, which is bandwidth-bound and degrades hard. That asymmetry is why a long-document workflow feels acceptable on offload right up until the model starts replying.

What happens when you run out of VRAM?

Offload is not a gentle fallback. Layers living in system memory are read across the PCIe bus at host memory bandwidth, and the arithmetic is unkind: dual-channel DDR4-3200 delivers roughly 51 GB/s in theory, against the 360 GB/s the RTX 3060 provides on-card (TechPowerUp). That is close to a 7× deficit before you account for transfer overhead.

A strong host helps at the margin. An AMD Ryzen 7 5800X — eight Zen 3 cores, sixteen threads — is the reference offload host for this class of build because it has enough cores to keep the offloaded layers busy and a memory controller that reaches DDR4-3200 comfortably. What it cannot do is change the bandwidth ceiling. The constraint is the memory bus, not the core count, which is why a faster CPU produces a smaller improvement than most builders expect.

Two practical consequences. First, populate both memory channels — a single DIMM roughly halves your offload bandwidth and is the most common self-inflicted wound in these builds. Second, treat offload as a way to try a model, not to live with one. If your daily driver is offloading, you have picked the wrong model for the card.

The hardware baseline

The MSI GeForce RTX 3060 Ventus 3X 12G OC is the reference card here for a specific reason: it is the cheapest honest 12 GB. The GA106 silicon underneath is not fast by 2026 standards — 3584 CUDA cores, a 192-bit bus, 360 GB/s — but the 12 GB capacity is the spec that decides whether a model loads, and capacity is the one thing this card has more of than several nominally faster 8 GB parts (TechPowerUp).

The 170 W TGP matters more than it looks. A local assistant is an always-on load, not a benchmark run, and 170 W is a figure you can leave running on a standard 550-650 W supply without a second thought. That combination — enough VRAM to hold a 14B model resident, low enough power to run continuously, and a used-market price that has stayed reasonable — is why this card keeps appearing in 12 GB local-LLM builds years after launch. A fuller treatment of the card is in the RTX 3060 12GB Local LLM Guide.

Context-length impact

The KV cache is the variable that turns a fit into a non-fit. It grows with context length, and on a card where weights already occupy most of the frame buffer, it is the growth that pushes you over.

The escape hatch is KV-cache quantization. Storing the cache at 8-bit instead of 16-bit roughly halves its footprint, and the runtimes expose it as a flag (llama.cpp). On a 12 GB card this is one of the highest-value settings available: it can be the difference between a 16K session that stays resident and one that spills. Quality impact on the cache is far less visible than quantizing weights an extra step down.

Common pitfalls on a 12 GB card

• Running single-channel RAM. Halves offload bandwidth. Costs nothing to fix and is the single most common configuration error in these builds.
• Quantizing below Q4 to force a fit. You are trading measurable output quality for a model you should not be running on this card. Pick the smaller model instead.
• Ignoring desktop VRAM overhead. A browser with hardware acceleration and a second monitor can consume several hundred megabytes of the frame buffer. On a card with 2.5 GB of margin, that is a meaningful slice — run the box headless if you can.
• Leaving KV cache at fp16 by default. Free headroom left on the table.
• Sizing storage for one model. Four quant variants of two models fill a 500 GB drive faster than expected.

Model library storage

Once you are comparing quants rather than downloading one, disk becomes the annoyance. Two Q4_K_M models, their Q5_K_M counterparts, a couple of experiments, and an OS will pass 500 GB without effort. A 1 TB Crucial BX500 1TB SATA SSD or a Kingston A400 960GB SATA SSD is the right answer here, and specifically the SATA answer.

The reason is that model files are read once per session and then live in VRAM or system RAM. The interface affects load time — seconds — and nothing at all about tokens per second afterward. Paying an NVMe premium for a model library buys you a faster startup and no faster inference. Capacity is what you want; buy the bigger SATA drive. The measurements behind that call are in NVMe vs SATA SSD for Local LLM Model Libraries.

Thermals for an always-on assistant

Token generation is a long, flat load rather than a spiky one, which changes what "adequate cooling" means. A cooler that survives a 10-minute benchmark can still let a box drift warm over a six-hour session, and the fan curve that was inaudible during a game becomes the loudest object in the room at 2 a.m.

For the host CPU, a Noctua NH-U12S is the quiet-air default: a single 120 mm tower with enough capacity for a 105 W-class part like the 5800X under sustained load, and a noise profile that suits a machine that never turns off. Pair it with case airflow that actually exhausts the GPU's 170 W rather than recirculating it — one intake and one exhaust, both slow, beats one fast fan. The air-versus-AIO comparison for this exact use case is in Cooling a 24/7 Local LLM Rig.

Perf-per-dollar and perf-per-watt

On a card in the 170 W class, the perf-per-watt story is straightforward: a resident model uses the GPU efficiently, and an offloaded model burns host CPU power to deliver fewer tokens. Offload is worse on both axes simultaneously — it is slower and it costs more watts per token, because the CPU is now doing work at DDR4 bandwidth that the GPU would have done at GDDR6 bandwidth.

The dollar axis points the same direction. The cheapest way to raise tokens per second on this build is not a faster card; it is choosing a model that stays resident. Phi-4 at Q4_K_M costs nothing over gpt-oss 20B and delivers full-GPU throughput. That is the best perf-per-dollar move available at this tier, and it is free.

Verdict matrix

Get gpt-oss 20B if… your prompts are short, your sessions are short, you want the mixture-of-experts model's behavior specifically, and you accept partial offload as the normal operating mode. It is a legitimate choice on a 12 GB card — just not a comfortable one.

Get Phi-4 14B if… you want an assistant that stays resident, holds a real context window, and answers at full GPU speed. This covers most people asking the question. Run Q4_K_M and enable 8-bit KV cache.

Get neither and step up to 24 GB if… you need long context on a 20B-class model, you want two models resident at once, or you have started reaching for Q3 quants to make things fit. That last one is the clearest signal that 12 GB is behind you.

Recommended pick

For a builder with one 12 GB card in 2026, run Phi-4 14B at Q4_K_M with an 8-bit KV cache. It is the only configuration in this comparison that keeps every layer on the GPU while leaving a usable context window, and staying resident is worth more day to day than the larger model's parameter count. Keep gpt-oss 20B on disk for short-prompt work where the MoE behavior earns its offload penalty.

Bottom line

The parameter count is the wrong number to shop on. What decides this comparison on a 12 GB card is what the weights leave behind, and Phi-4 at Q4_K_M leaves enough to work with while gpt-oss 20B does not. Both models are worth having; only one of them is worth making your default on this hardware. If your workload genuinely needs the 20B model at long context, that is not a quantization problem — it is a 24 GB problem, and the honest answer is to budget for the card rather than fight the frame buffer.

Related guides

• RTX 3060 12GB Local LLM Guide: Which Models Actually Fit
• Best GPU for Local LLMs Under $350
• llama.cpp vs Ollama for Single-User Chat on an RTX 3060
• Quantization on a 12GB GPU: q4 vs q5 vs q8
• i7-9700K vs Ryzen 7 5800X as a CPU-Offload Host

Citations and sources

• OpenAI — Introducing gpt-oss — accessed 2026-09-04
• openai/gpt-oss-20b model card — accessed 2026-09-04
• Phi-4 Technical Report (arXiv 2412.08905) — accessed 2026-09-04
• TechPowerUp — GeForce RTX 3060 12 GB specifications — accessed 2026-09-04
• llama.cpp — quantization and offload documentation — accessed 2026-09-04

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.