Qwen3-Coder-Next 80B-A3B on a 12GB RTX 3060: Per-Quant VRAM, RAM and tok/s

Introduction

Qwen3-Coder-Next is an 80B-parameter model that activates only about 3B parameters per token. Per the Qwen3-Coder-Next model card, it has 48 layers and 512 experts, with 10 routed experts plus 1 shared expert per token and a native 262,144-token context. That split changes what a 12GB card can do. A dense 70B model at Q4 has to stream about 40 GB of weights for every token it generates. Qwen3-Coder-Next reads roughly 1.5B routed-expert parameters per token, and only those reads have to cross into system RAM.

This guide is for budget local-coding-agent builders on AM4, meaning a Ryzen 5000 CPU, DDR4 and a 12GB Ampere card. You want an agentic coder in Cline, Qwen Code or Claude Code that runs on hardware you already own or can buy used. As of October 2026, nobody has published a clean, reproducible RTX 3060 benchmark of this exact model. So this page gives you the following:

• File sizes, taken from the GGUF repo.
• Memory math, taken from the model's own config.json.
• The closest published 12GB-card measurements, each attributed to its source.
• Where an estimate is unavoidable, the arithmetic behind it, so you can check it.

Key Takeaways

• No quant fits in VRAM. The smallest sensible quant, UD-IQ3_XXS, is 28.5 GB. Q4_K_M is 48.5 GB (unsloth GGUF repo). Every RTX 3060 setup is therefore an offload setup.
• The closest published 12GB result is about 40 tok/s generation and about 500 tok/s prefill. It came from an RTX 4070 12GB with 64GB of DDR5, using --fit and a q8_0 KV cache at 64K context (carteakey.dev write-up).
• DDR4 is the limiter on AM4. In llama.cpp issue #19480, one user reported 7 tok/s at Q4_K_M in pure CPU mode on DDR4-3200. GPU offload exists to beat that number.
• The KV cache is unusually cheap. Only 12 of the 48 layers use full attention, with 2 KV heads × 256 dims. That works out to about 24 KB per token at f16, or about 1.6 GB at 64K context.
• Budget 64GB of RAM for Q4_K_M. Budget 32GB only if you accept 2-bit quants.

Per-quant table: GGUF size, VRAM used, system RAM needed, generation tok/s

File sizes come from the unsloth/Qwen3-Coder-Next-GGUF file listing, in decimal GB, read October 2026. The VRAM column assumes llama.cpp's --fit fills the card to within about 512 MiB, which is how the 12GB RTX 4070 setup cited below was configured.

The system-RAM figure is the file size minus about 9 GB resident on the GPU, plus about 8 GB for the OS, your editor and the agent's tooling. It is rounded up to a real DIMM kit size.

No public source reports a generation speed for this model on an RTX 3060 specifically, so the last column says so. Nothing has been filled in. Here are the nearest published reference runs:

SpecPicks estimate, not a measurement: for an RTX 3060 with dual-channel DDR4-3200, expect roughly 15–22 tok/s generation at Q4_K_M. The estimate takes the RTX 4070's 33.7–40 tok/s and scales it by the system-memory bandwidth ratio. DDR4-3200 dual channel is 51.2 GB/s theoretical, against 96 GB/s for DDR5-6000, a ratio of about 0.53. Routed-expert reads from system RAM dominate generation time under offload, which is why that ratio is the right scaling factor. The 3060's own 360 GB/s of GDDR6 is slower than the 4070's 504 GB/s, but that only affects the part of each token computed on the GPU. If you run the model and get a number, the llama-bench command below produces figures you can compare directly with the reference rows.

On quality, the unsloth guide reports two third-party results. On Aider Polyglot, UD-IQ3_XXS comes close to BF16. On LiveCodeBench v6, Q3_K_M trails Q4_K_M. Unsloth's advice is to use at least Q4_K_M. In the issue #19480 perplexity table, perplexity is 8.38 at Q4_K_M, 8.25 at Q5_K_M and 8.23 at Q6_K, so the curve is nearly flat above Q5.

How does MoE expert offload keep the active weights on a 12GB card?

Each of the 48 layers has two parts. The first is the mixer: Gated DeltaNet linear attention in 36 layers and full gated attention in 12. The second is an MoE block with 512 small experts, each a 2048→512→2048 gated MLP of about 3.1M parameters.

The experts make up almost the entire model. 512 experts × 48 layers × 3.1M parameters is about 77B of the 80B total. The mixers, the shared experts, the router and the embeddings make up the remaining few billion. Those are the weights every token touches, so they belong on the GPU.

llama.cpp gives you three ways to split the model:

• --n-cpu-moe N keeps the expert tensors of the first N layers in system RAM and everything else on the GPU. Start high, around 40, and lower N until VRAM is nearly full.
• -ot / --override-tensor uses a regex. -ot ".ffn_.*_exps.=CPU" pushes every routed expert to the CPU. It is the simplest option, but it is also the slowest, at 33.7 tok/s against 39.9 on the 4070 rig, because it leaves VRAM unused.
• --fit on is the newest option. llama.cpp measures free VRAM and places as many whole layers and expert tensors on the GPU as fit, keeping a margin you set with --fit-target. On the 4070, --fit beat a hand-tuned --n-cpu-moe 40 on prefill, at 511 against 451 tok/s, and matched it on generation.

A starting command for an RTX 3060 with 64GB of RAM:

llama-server -m Qwen3-Coder-Next-Q4_K_M.gguf \
 --ctx-size 65536 --fit on --fit-target 512 \
 -ctk q8_0 -ctv q8_0 --flash-attn on \
 --threads 8 --threads-batch 16 --no-mmap --jinja \
 --temp 1.0 --top-p 0.95 --top-k 40 --min-p 0.01

The sampling values are the ones the unsloth guide recommends. Set --threads to your physical core count, which is 8 on a 5800X and 6 on a 5600X. In issue #19480, SMT threads hurt generation.

Update llama.cpp before you start. The DeltaNet layers ran on a slow generic path until native Gated DeltaNet kernels for CPU, CUDA, Metal and Vulkan landed in March 2026 (PR #20334, referenced in issue #19480). Builds from before that release explain most of the "5x slower than expected" reports.

How much system RAM and which CPU do you need?

Under offload, the CPU's main job is reading about 1.5B routed-expert parameters per generated token. At Q4_K_M, about 4.9 bits per weight, that is roughly 0.9 GB per token. It means memory bandwidth, not core count, sets the ceiling.

Both chips have the same memory controller and the same 51.2 GB/s theoretical bandwidth at DDR4-3200, so generation speeds should land close together. The 5800X's two extra cores help in two places. One is prefill threads (--threads-batch). The other is running a build, a test suite or a language server next to the model without stealing cycles from inference. Specs are from AMD's Ryzen 7 5800X product page.

Rules that matter more than the CPU choice:

1. Install two or four matched DIMMs. A single stick halves bandwidth and roughly halves offloaded generation speed.
2. Turn on XMP/DOCP. The carteakey.dev author ran DDR5-6000 at 2000 MT/s for months because the BIOS was set to Auto, a 3x handicap. AM4 boards default to DDR4-2133 the same way.
3. Size RAM to the quant. Use the table above. If the model does not fit in RAM plus VRAM, llama.cpp pages from disk and generation collapses.

What does long context cost?

This model's KV cache is far smaller than most 80B-class models', because only the 12 full-attention layers keep one. From config.json, each of those layers has 2 KV heads × 256 head dims. At f16, that is 12 × 2 (K+V) × 2 × 256 × 2 bytes, or about 24 KB per token. The 36 DeltaNet layers keep a fixed-size recurrent state of under 100 MB that does not grow with context.

These figures are calculated from the published architecture. Real VRAM use comes out higher, because llama.cpp's compute buffers also grow with context and batch size. The carteakey.dev author reported about 4 GB at f16 and about 2 GB at q8_0 for 64K once everything is counted.

Either way, the trade is the same. Every gigabyte the cache takes is a gigabyte of expert tensors that move to the slower CPU side. q8_0 KV quantization roughly halves the cache with no visible quality change at short contexts, per the same write-up. For a 12GB card, 64K with a q8_0 cache is a sensible default. 128K works but pushes another layer or two off the GPU.

Prefill vs generation: why prompt processing drops more than token generation under offload

Generation handles one token at a time, so each step reads only the 10 active experts per layer. That load is small and bandwidth-bound.

Prefill processes hundreds or thousands of prompt tokens per batch, and across a large batch nearly all 512 experts in every layer get used. llama.cpp can compute those batches on the GPU even when the weights live in system RAM, but it has to copy the expert weights across PCIe to do it. On a 12GB card, prefill speed therefore depends on three things:

• PCIe link. The RTX 3060 is a PCIe 4.0 x16 card. On a B450 or A520 board the link drops to 3.0, which halves transfer bandwidth.
• Batch size. A larger --ubatch-size spreads each weight copy across more tokens.
• How many layers stay fully resident. This is why --fit won on prefill on the 4070 rig.

Agent loops make this matter. Every tool call resends a long system prompt, the file context and the tool output, so prefill time, not generation speed, often decides whether the agent feels responsive. Turn on prompt caching (--cache-reuse in llama-server) so unchanged prefixes are not reprocessed.

Which RTX 3060 12GB card to buy

All three cards use the same GA106 GPU with 3,584 CUDA cores, 12GB of GDDR6 at 15 Gbps on a 192-bit bus (360 GB/s), a 170 W reference board power and a PCIe 4.0 x16 interface (TechPowerUp RTX 3060 12 GB spec sheet). For LLM inference they perform the same. The differences are size and cooling.

Specs are from the manufacturers' spec sheets. Check your exact SKU and revision before buying if case clearance is tight. Factory overclocks make no meaningful difference here. Offloaded inference is limited by memory and PCIe, not shader clock. Choose the cooler that fits your case and your tolerance for fan noise. A 3-fan cooler generally holds lower fan speeds at the same load.

Perf-per-dollar: is stepping up to 16GB or 24GB worth it?

The RTX 3060 12GB launched at a $329 MSRP. Using the estimated 15–22 tok/s, that is about 0.045–0.067 tok/s per launch-MSRP dollar. The RTX 4070 12GB launched at $599 and reached about 40 tok/s on a DDR5 platform, about 0.067 tok/s per dollar. Its advantage comes largely from the faster RAM behind it, not from the card itself.

Going to 16GB keeps more expert layers resident. The issue #19480 data shows a 16GB card reaching 25–36 tok/s with DDR5. Going to 24GB, such as an RTX 3090, moves roughly 40% of Q4_K_M onto the GPU and improves prefill the most.

The upgrade order for this model on AM4:

1. Install 64GB of dual-channel RAM.
2. Confirm XMP is on.
3. Make sure the board runs the card at PCIe 4.0.
4. Only then think about a larger GPU.

For per-model VRAM sizing across the 7B–70B range, see the best local LLM for 12GB VRAM guide. For a smaller MoE that fits more comfortably, see the Qwen3.6-35B-A3B on RTX 3060 deep dive.

Worked examples

Example 1: 32GB of RAM, unwilling to buy more. Q2_K (29.2 GB) or UD-IQ3_XXS (28.5 GB) loads with about 9 GB on the GPU, leaving about 20 GB in RAM. That is tight with a browser open. Choose UD-IQ3_XXS, which unsloth's Aider data rates close to BF16. Keep context at 32K and close other memory-heavy apps.

Example 2: 64GB DDR4-3600, 5800X, Ventus 2X. Run Q4_K_M with --fit on, 64K context and a q8_0 KV cache. This is the configuration the estimated 15–22 tok/s applies to. The slightly faster RAM may lift it toward the top of that range. It is a good fit for single-repo edits in Cline or Qwen Code.

Example 3: 128GB of RAM, wanting near-lossless output. Q6_K (65.6 GB) fits in RAM with room to spare. Generation slows roughly in proportion to bytes read per token, about 35% more data than Q4_K_M. The perplexity gain over Q5_K_M is 0.02. For most people, Q5_K_M is the ceiling worth running on this card.

Common pitfalls

• Old llama.cpp builds. Builds from before March 2026 lack the native Gated DeltaNet kernels and run several times slower. Update first, benchmark second.
• Single-channel RAM. One 32GB stick plus one 16GB stick, or a lone DIMM, cuts bandwidth and generation speed.
• No VRAM margin. The carteakey.dev author moved from a 128 MiB to a 512 MiB --fit-target because the CUDA memory pool grows in 1 GiB chunks and caused out-of-memory errors as prompts grew.
• Using -ot for all experts. It is simple, but it leaves VRAM idle. --fit or a tuned --n-cpu-moe is faster on both prefill and generation.
• Expecting thinking mode. The model card states that this model is non-thinking only and never emits <think> blocks. Don't configure your agent to wait for them.

Bottom line: when this setup is right, and when it isn't

It's right if you have an AM4 box and an RTX 3060 12GB already, or want the lowest-cost route to an 80B-class agentic coder, and your work is single-file edits, test writing, and repo Q&A under about 64K tokens. Spend on RAM first.

It isn't right for all-day agentic refactors that regularly pass 100K tokens of context. At that size, every tool call pays a long prefill penalty over PCIe, the KV cache competes with expert layers for 12GB, and the loop slows down. A 24GB card, a 16GB card on a DDR5 platform, or a unified-memory system with 64GB or more keeps more of the model resident. If you mostly want fast autocomplete, a dense 7–9B coder that fits entirely in VRAM is faster and simpler. See the local coding agent on RTX 3060 guide.

Related guides

• Running a local coding agent on an RTX 3060 12GB
• Best local LLM for 12GB VRAM
• Qwen3.6-35B-A3B MoE on an RTX 3060 12GB
• 32K-context VRAM budget on an RTX 3060 12GB
• How much system RAM to run Llama 70B with an RTX 3060
• Qwen 3.6 27B quantization benchmarks

Citations and sources

All sources were accessed October 26, 2026.

• Qwen/Qwen3-Coder-Next model card: architecture, parameter counts, context length and non-thinking mode.
• unsloth/Qwen3-Coder-Next-GGUF: per-quant GGUF file sizes.
• Unsloth: Qwen3-Coder-Next how to run locally: memory guidance, sampling parameters and third-party quant quality results.
• carteakey.dev: Running Qwen3-Coder-Next at 40 t/s on consumer hardware: RTX 4070 12GB offload measurements and --fit tuning.
• llama.cpp issue #19480: community CPU and partial-offload measurements and the Gated DeltaNet kernel fix.
• TechPowerUp RTX 3060 12 GB spec sheet: GPU specifications.
• AMD Ryzen 7 5800X product page: CPU specifications.

This page is an editorial synthesis. Its sources are published specifications and third-party measurements, plus arithmetic labelled as estimates. SpecPicks did not benchmark this configuration.