Which LLMs Fit a 12GB RTX 3060? Per-Model VRAM Cheat Sheet (2026)

A 12 GB RTX 3060 fits every 7B–8B model comfortably at 4-bit quantization, most 13B–14B models with a modest context, and 27B–32B models only with aggressive q3/q2 quantization or partial CPU offload. As of 2026 the practical local-LLM shortlist on this card is: Llama-3.1-8B, Mistral-7B/Nemo, Qwen2.5 7B–14B, Phi-3.5, Gemma-2 9B, and Mixtral-8x7B at q3 with offload — plus almost anything smaller.

Why per-model VRAM math matters for the 12GB tier

The ZOTAC RTX 3060 12GB and MSI RTX 3060 Ventus 2X 12G sit at a specific inflection point in the local-LLM market. They ship with more VRAM than the RTX 3060 Ti (8 GB), RTX 4060 (8 GB), and RTX 4060 Ti 8 GB — all cards that look faster on paper but choke on any model that spills past 8 GB. They cost far less than a 16 GB RTX 4060 Ti or a 16 GB RTX 4080, and they're still available new and used at prices most home builders can stomach. That combination — decent GDDR6 bandwidth, actual room for real models, and a used market flooded with cards from the crypto exodus — makes the 12 GB tier the single most useful budget for local inference in 2026.

But the moment you go shopping for a model to run, the tutorials get vague. Everyone says "yes, 12 GB is fine for 7B" and then hand-waves at 13B, 34B, and 70B. That's not useful when you're trying to plan a workflow. This guide answers the actual question: for each common model size and quant, do the weights and KV cache fit inside a 12 GB card, at what context length, and at what tok/s. If your workload is "one user, one card", these are the numbers you need.

Key takeaways

• 7B–8B q4_K_M models fully fit with plenty of headroom for long context.
• 13B–14B q4_K_M models fit at 4K–8K context but get tight at 32K.
• 27B–32B models need q3 or q2, or partial CPU offload; usable but not fast.
• 70B on a 12 GB card is not practical (see the offload reality-check guide).
• KV cache growth is the silent VRAM killer at long contexts.
• q4_K_M is the default; q5_K_M when quality matters and you have headroom.

How do you calculate VRAM needed for a model at a given quant?

Rough formula for planning:

VRAM ≈ (parameters × bytes_per_weight) + KV_cache + framework_overhead

Bytes-per-weight for common quants (effective, including overhead):

• q2_K → ~0.35 bytes/param
• q3_K_M → ~0.45 bytes/param
• q4_K_M → ~0.55 bytes/param
• q5_K_M → ~0.65 bytes/param
• q6_K → ~0.80 bytes/param
• q8_0 → ~1.10 bytes/param
• fp16 → 2.00 bytes/param

KV cache per token, per layer ≈ 2 × hidden_size × bytes(kv_quant). At fp16 KV cache, that's a few kilobytes per token per layer — a 32-layer model at 4K context runs to roughly 500 MB. At 32K context it's 4 GB, which can single-handedly evict a 13B model.

Add 1–2 GB for framework overhead (CUDA context, activation buffers) to stay safe inside 12 GB, per the Hugging Face LLM optimization guide.

Which 7B/8B models fit fully on a 12GB RTX 3060?

All of them, at q4_K_M, with room for long contexts. Concrete footprints:

Every entry above still leaves 3–5 GB of VRAM headroom, so you can push context to 32K on any of them without spilling.

Can 13B-14B models fit, and at what quant + context?

Yes, with tighter margins. At q4_K_M a 13B model uses ~8 GB for weights alone; at 4K context the KV cache is under a gigabyte, so total footprint sits around 9 GB and 12 GB of VRAM is comfortable. At 32K context the KV cache balloons to 2+ GB and you're at the edge — closing browser tabs and dropping to q3_K_M becomes worthwhile.

Tok/s on 13B at q4 lands around 25–40 on the 3060 — noticeably slower than 7B but still faster than reading.

What about 27B-32B with partial CPU offload?

Yes, with meaningful trade-offs. Models like Gemma-2-27B and Qwen2.5-32B fit only partially in 12 GB even at q4, so llama.cpp keeps some layers in VRAM and streams the rest from system RAM through the PCIe bus. Generation tok/s drops sharply — think 4–10 tok/s depending on how many layers stay in VRAM.

Aggressive q3_K_M brings a 32B model down to about 14 GB, so with 8 GB in VRAM and 6 GB spilled to RAM you can run it. It works. It's not fast. Reserve this for tasks where you specifically need the reasoning of a bigger model and can tolerate 5-second-per-token wait times.

How does context length eat into your 12GB budget?

Every model layer stores a K and V vector per token in the cache. For a 32-layer, 4096-hidden-dim model at fp16 KV: each token = 2 × 4096 × 2 bytes × 32 layers = 512 KB. That means:

• 4K context → 2 GB KV cache
• 8K context → 4 GB KV cache
• 16K context → 8 GB KV cache
• 32K context → 16 GB KV cache — larger than the entire card

You can drop KV to q8_0 (halves cache) or q4_0 (quarters it) in llama.cpp, per the ggml-org/llama.cpp docs. Quantized KV caches lose a small amount of quality but reclaim gigabytes; on a 12 GB card at long context, they're often the difference between "fits" and "OOM".

Spec table: model size vs quant vs VRAM footprint vs fits-on-3060

Weight footprint only — add KV cache and framework overhead. Y fits fully, tight fits with limited context, N requires offload.

Benchmark table: tok/s on RTX 3060 12GB

Numbers from llama.cpp community benchmarks; the TechPowerUp RTX 3060 spec page confirms the underlying 360 GB/s memory bandwidth that dominates these results.

Quantization matrix

Context-length impact analysis

Context is where 12 GB starts to feel small. Practical guidance:

• 4K context — everyone fits, don't worry.
• 8K context — 13B q4 is comfortable; 7B has huge headroom.
• 16K context — 13B needs KV quantization; 7B is fine.
• 32K context — 7B q4 is comfortable; 13B needs q3 or KV-q4.
• 128K context — only 3B-class models fit end-to-end without heavy KV quantization.

Perf-per-dollar math vs stepping to a 16GB card

Used RTX 3060 12GB cards trade around $200–$260 in mid-2026 US markets. New MSI Ventus units sit near $500 retail. A 16 GB RTX 4060 Ti runs $450 new; a 16 GB RTX 4080 Super is $999+. If your target model list stops at 13B, the 3060 is a runaway win on dollars per usable VRAM. Once you want 27B+ comfortably or need FLUX/Stable Diffusion throughput, the 4060 Ti 16 GB or a used 3090 24 GB (~$750) becomes the honest recommendation.

For a starter build, pairing a 3060 with a value CPU like the AMD Ryzen 5 5600G or the Ryzen 7 5800X, and a WD Blue SN550 1TB NVMe for model storage, gives you a competent full local-LLM host under $800 total.

Bottom line: the practical 12GB model shortlist

For daily use on a 12 GB RTX 3060, the shortlist as of 2026 is:

• Chat/general — Llama-3.1-8B, Qwen2.5-7B, Mistral-7B — all q4_K_M
• Long context — Phi-3.5-mini, Mistral-Nemo — smaller models, more headroom
• Coding — Qwen2.5-Coder-14B, CodeLlama-13B — q4_K_M, 4K–8K ctx
• Reasoning-heavy — Qwen2.5-14B q4_K_M, or Qwen2.5-32B q3 with offload if you can wait
• Speed matters most — Phi-3.5-mini or Mistral-7B q4

Related guides

• Can a 12GB RTX 3060 run a 70B LLM? — the offload reality check
• llama.cpp vs Ollama vs vLLM on a 12GB 3060 — runtime choice
• Run Local LLMs on a Ryzen 5 5600G — the CPU-only alternative

Sources

• TechPowerUp RTX 3060 specs — official memory bandwidth and CUDA-core counts
• ggml-org/llama.cpp — quantization docs and KV cache internals
• Hugging Face LLM optimization guide — memory math and caching techniques