Qwen 3.8 Open Weights: What Fits on a 12GB GPU

A 12GB card such as the RTX 3060 comfortably runs a Qwen 3.8 checkpoint in the 7B–14B parameter range at a four-bit-class quantization (q4_K_M) with an 8K context window and around 2GB of headroom for the KV cache. Larger Qwen 3.8 checkpoints — the 32B and MoE variants — do not fit in 12GB without CPU offload, and offloading collapses decode throughput. Size the checkpoint to the card first, then pick the quantization.

Who this piece is for, and why Alibaba's cadence matters

Alibaba's Qwen family has become the default reference point for anyone running a local coding or reasoning agent on a single consumer GPU. Per public Hugging Face releases, the team has now shipped four generational bumps in roughly eighteen months, each carrying a matched set of dense checkpoints from 1.5B through 32B parameters plus a mixture-of-experts variant. Qwen 3.8 is the newest of these, landing in the same window as Moonshot's Kimi K3 and framed by community coverage as the direct answer to it.

For readers with a single 12GB card — an RTX 3060 12GB is the reference platform for this article — the practical question is not "is Qwen 3.8 state of the art?" but "which Qwen 3.8 checkpoint fits my card without crippling context length or throughput?" That is the question we answer here. Numbers below are pulled from public benchmark reports, community measurements on llama.cpp and vLLM harnesses, and the TechPowerUp GPU database for hardware specs — not from a first-party testbench in our lab.

We are also assuming the reader owns, or is about to buy, roughly the rig Alibaba's Qwen team has been describing as a "good enough" single-user local host in 2026: an entry-tier 12GB card (MSI GeForce RTX 3060 Ventus 3X 12G), an eight-core AM4 host CPU (AMD Ryzen 7 5700X), a fast NVMe drive for the working checkpoint (WD Blue SN550 1TB), and a cheaper SATA SSD as the archive shelf (Crucial BX500 1TB). Prices vary by day; treat all dollar figures below as of July 2026.

Key takeaways

• q4_K_M is the default for 12GB. It halves memory versus q8 with limited perplexity loss on public evals — start there and only step up or down when the profile demands it.
• KV cache is the real ceiling. At 32K context, KV growth eats several GB even at four-bit weight quantization. Cut context before cutting quant if you need to fit more model.
• Offload is a trap for decode. As soon as a layer lives in system RAM, generation is bound by CPU-to-DDR bandwidth (~50GB/s), not by GDDR6 (~360GB/s). A single offloaded layer can drop decode by 3–5×.
• NVMe matters for load time, not inference. A ~15GB checkpoint reads in ~7 seconds off an SN550 versus 25+ seconds off a SATA drive — meaningful when you swap models often.
• Perf-per-dollar breaks even fast for agent-loop users; it does not break even for occasional chat. Do the arithmetic with your actual monthly token count before buying hardware.

What did Alibaba actually release, and when?

Per the Qwen team's public model cards on Hugging Face, Qwen 3.8 shipped as a family rather than a single model. The lineup follows the pattern established in earlier Qwen generations: dense checkpoints at 1.5B, 3B, 7B, 14B and 32B parameters, plus an MoE variant with a larger nominal parameter count and roughly 3–4B active per token. Licensing continues to be permissive for research and most commercial use, following the Qwen custom license terms — worth reading in full before shipping a product that redistributes weights.

The release itself lands mid-2026, roughly ten days ahead of the Kimi K3 open-weight announcement from Moonshot. Community measurements published to the llama.cpp GitHub discussions started appearing within 72 hours of upload, which is typical for a permissive Qwen drop. Numbers below reference those community measurements — they are not first-party specpicks benchmarks — and are consistent within roughly ±15% across independent posts.

Which Qwen 3.8 size fits a 12GB card?

The RTX 3060 12GB has 12,288MB of VRAM, of which roughly 400–600MB is reserved by the driver and any concurrent desktop workload. That leaves ~11.5GB of usable memory for model weights + KV cache + activation buffers. The quantization matrix below is what community reports observe for the Qwen 3.8 7B and 14B dense checkpoints on that budget.

These are steady-state decode figures from llama.cpp with a fully-loaded model. Real prompt processing (prefill) is separate and covered below. Numbers vary meaningfully with driver version, kernel, and whether flash-attention is compiled in; treat them as an ordering, not a promise.

Spec-delta table: Qwen 3.8 vs Kimi K3 vs Qwen 3 prior generation

Reported coding-index numbers come from Artificial Analysis, which bundles harness + tool schema + retry policy into its published scores. Swap any of those and the results shift measurably, which is why the "reported" hedge matters.

Prefill vs generation: why bandwidth caps decode but not prompt ingestion

The RTX 3060's 12GB variant runs on a 192-bit GDDR6 memory interface at 15Gbps effective, which per TechPowerUp is roughly 360GB/s of theoretical bandwidth. For token generation — where the model reads all of its weights, per-token, to produce a single next token — this bandwidth is the direct throughput cap. A four-bit 7B model at ~4.4GB weights and a 42–50 tok/s decode figure sits right where the arithmetic predicts (bandwidth ÷ weight size, minus overhead).

Prefill is a different story. During prompt ingestion the model can amortize a weight-read across many tokens in the same batch, converting a bandwidth-bound problem into a compute-bound one. The 3060 has 3,584 CUDA cores and ~13 TFLOPS of FP16 compute; prefill speeds of 800–1,400 tok/s on 7B q4_K_M are typical, per community reports. This asymmetry — fast prefill, slow decode — is why agent workloads (which stream tool output back into the context) can feel slow even though the raw arithmetic shows the card should be able to keep up. You are paying decode cost on every new token the agent has to inspect.

KV-cache growth at 8K, 32K and 128K context

Weight-quantization compresses the model itself; the KV cache does not shrink with weight quant unless you also apply a KV cache quantization pass (widely supported in llama.cpp as of the 2026 releases). Rough per-context KV budgets for the Qwen 3.8 7B checkpoint at fp16 KV:

For the 14B checkpoint the KV numbers roughly double per context step. Any 14B use on 12GB is realistically limited to 8K context with q4_K_M weights.

The hardware you actually need

The MSI GeForce RTX 3060 Ventus 3X 12G remains the entry-price ticket to running Qwen 3.8-class models locally in 2026. It is a three-fan overclocked variant of the reference 3060, with a 170W TGP, 12GB of GDDR6, and — critical for local LLM work — the full 192-bit memory bus that budget cut-down 3060 variants sometimes retain and sometimes don't (check the SKU number, not the marketing name).

For the host CPU, the AMD Ryzen 7 5700X is the sweet-spot AM4 pick: eight cores, sixteen threads, a 65W TDP, and enough single-thread performance to keep the sampling loop and tokenizer off your desktop cores. If you plan to offload any layers to system RAM, prioritize dual-channel DDR4-3600 CL16 or better — CPU memory bandwidth becomes the direct throughput limiter once a single layer leaves VRAM.

Storage tiering matters. The WD Blue SN550 1TB NVMe is the reference "hot checkpoint" drive: a Gen3 x4 PCIe part rated at 2,400MB/s sequential read, which turns a 15GB checkpoint load into a ~7-second operation. Cheaper SATA capacity — the Crucial BX500 1TB SATA SSD at ~540MB/s — is the archive shelf for the checkpoints you swap in occasionally. Do not try to run inference directly from the SATA drive; the load-time gap is felt every time you switch models.

Multi-GPU: does a second 12GB card help?

Adding a second RTX 3060 12GB doubles your available VRAM to 24GB, which unlocks 14B checkpoints at q6/q8 or 32B checkpoints at aggressive quantization. It does not help single-stream decode throughput in a meaningful way, because Qwen 3.8's dense checkpoints do not benefit from tensor-parallelism at consumer-GPU scale — the interconnect between the cards (PCIe 4.0 x16, or worse if you're on a consumer AM4 board that splits to x8/x8) becomes the bottleneck versus the on-card memory bus.

The absence of NVLink on the 3060 makes this worse than it would be on data-center parts. Community benchmarks on dual-3060 rigs consistently show that pipeline-parallelism (splitting the model layer-wise across the two cards, one card processing after the other) gives roughly the same decode speed as a single card, plus the model just fits. That is often enough — you get capacity you didn't have — but do not expect a 2× throughput improvement.

Where a second card does help: batched inference (multiple concurrent conversations), and running two different models simultaneously (a small one for embedding, a larger one for the main agent).

Perf-per-dollar and perf-per-watt vs hosted API

At July 2026 street prices, an MSI 3060 Ventus is roughly $300, a Ryzen 7 5700X is roughly $150, a SN550 1TB is roughly $60, and a BX500 1TB is roughly $55. Add a basic AM4 board, 32GB of DDR4-3600, a 650W power supply, and a case — call the total build cost around $850–1,000 depending on part selection.

For agent workloads that consume 30–50M prompt tokens and generate 3–5M output tokens per month, hosted API pricing (roughly $0.30–0.60 per million prompt tokens and $1–2 per million output tokens for open-weight models via commodity providers, per current listings) works out to $15–40 per month. The rig pays for itself in 2–4 years of continuous use — not fast, but competitive if you value latency or privacy.

For agent loops that re-ingest large repository contexts many times per day, monthly token counts of 500M+ are common, and the payback drops below a year. That is where local starts to make undeniable financial sense.

Power draw is the other axis: an RTX 3060 pulls 170W under load and roughly 15W at idle. At $0.15/kWh, an eight-hour-a-day agent loop is roughly $6/month in electricity — negligible against the API cost it replaces.

Verdict matrix

Get a 12GB card (RTX 3060 12GB) if… you want to run 7B–14B Qwen 3.8 checkpoints for personal coding, chat, or agent work; you accept q4_K_M quantization; your context ceiling is 8K–32K; and your monthly token budget on hosted APIs already exceeds ~$20.

Step up to 24GB (RTX 4090 / RTX 3090 / dual 3060) if… you need the 32B checkpoint at usable quantization; you want 128K context without offload; or you plan to run agent loops in parallel.

Stay on hosted API if… your daily token count is small; you need the largest checkpoint at fp16; you cannot amortize a ~$1000 build over at least a year; or your workload is bursty enough that a spun-up API call beats a warm local model.

Common pitfalls

• Buying a 3060 8GB variant by accident. Some SKUs share model families with the 12GB cards but ship an 8GB, 128-bit memory config. The Ventus 3X 12G explicitly carries the 12GB spec — verify the SKU string, not just the family name, before checkout.
• Ignoring flash-attention. Compiling llama.cpp with flash-attention support cuts KV memory pressure roughly in half at long contexts on the 3060, at the cost of a modest tok/s dip on short contexts. The tradeoff is almost always worth it.
• Running the model off SATA. A SATA SSD is fine for cold storage; running inference off it costs you 5–15 seconds of load time every swap. Put the working checkpoint on NVMe.
• Trusting single-run tok/s numbers. GPU thermal throttling on the 3060 kicks in around 83°C. A 30-minute sustained run will be 5–15% slower than the first-minute number if case airflow is poor.

Bottom line

Qwen 3.8 on a 12GB RTX 3060 is the cheapest useful path into serious local-LLM work in 2026. The 7B checkpoint at q4_K_M with an 8K context is comfortable, useful, and reasonably close to the frontier for that model size. The 14B checkpoint fits at q4_K_M with the context window tuned down, and rewards the effort with noticeably better reasoning. The 32B checkpoint does not fit and is not worth pretending otherwise — either buy a 24GB card, or stay on hosted API for the sizes you actually need.

Whichever checkpoint you pick, do the arithmetic with your own monthly token count before buying hardware. The break-even is real, but it is a function of your workload, not a headline.

Related guides

• Best 27-inch 4K Monitor for PC and Console
• Kimi K3 vs Qwen 3.8 for Local Coding Agents in 2026
• vLLM vs Ollama for a Single-User 12GB Rig in 2026
• Ryzen 5 5600G vs Ryzen 7 5700X for a 24/7 LLM Server

Citations and sources

• Hugging Face — Qwen organization page
• TechPowerUp — GeForce RTX 3060 12GB specifications
• llama.cpp — GitHub Discussions (community benchmarks)
• Artificial Analysis — Coding agents leaderboard

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.