Running Qwen3.5/3.6 with NextN MTP on an RTX 3090 Ti

Multi-token prediction (MTP) — sometimes labeled "NextN" in checkpoint and code comments — lets a model propose more than one future token per forward pass using an extra prediction head trained into the checkpoint itself. Paired with llama.cpp's speculative decoding path, it can reduce the number of full forward passes needed to generate a response, without requiring a second, separate draft model sitting in VRAM. On a single RTX 3090 Ti — a 24GB card that's now several generations behind the current flagship but still one of the most common high-VRAM cards in enthusiast rigs — that VRAM efficiency is the whole appeal: self-speculative decoding buys some of the benefit of speculative decoding without the memory tax of hosting a second model.

This guide walks through what MTP actually is, whether a 3090 Ti has the headroom to run Qwen3.5 or Qwen3.6 with it enabled, and how to approach setup in llama.cpp. It intentionally avoids quoting specific tokens-per-second figures for this exact combination, because no verifiable public benchmark for Qwen3.5/3.6 MTP decoding on a 3090 Ti was available to cite at the time of writing — throughput here is genuinely workload- and build-dependent, and any number without a source should be treated with suspicion.

What NextN MTP actually is

Multi-token prediction was popularized as an architectural feature of DeepSeek-V3, where the model is trained with an additional module that predicts several tokens ahead rather than just the next one. At inference time, that extra head can double as a self-speculative decoder: the model drafts a short run of candidate tokens using its own MTP head, then the main decoding path verifies them in a single batched forward pass, accepting the ones that match what full decoding would have produced. Tokens that get accepted are effectively "free" — generated without a dedicated forward pass each.

This is architecturally different from classic speculative decoding, which pairs a large target model with a smaller, independently-trained draft model (for example, a 1B model drafting for a 70B target). Self-speculative MTP skips the second model entirely, which is what makes it attractive on a VRAM-constrained single-GPU setup like a 3090 Ti — you're not paying rent on a second checkpoint just to get a speedup.

Whether Qwen3.5 or Qwen3.6 checkpoints ship this head depends on the specific release and quantization; check the model card on Qwen's Hugging Face org page and confirm your llama.cpp build actually supports loading and using it — support for MTP/NextN-style heads has landed in llama.cpp incrementally, and flag names have shifted across releases, so llama-server --help (or llama-cli --help) on your specific build is the source of truth, not any single article.

Does an RTX 3090 Ti have enough VRAM?

The RTX 3090 Ti launched with 24GB of GDDR6X, a 384-bit memory bus, and roughly 10,752 CUDA cores, at a 450W TDP — specs confirmed on NVIDIA's own product page. That 24GB ceiling is the single biggest constraint for anyone trying to run a large MoE model like Qwen3.6-35B-A3B locally.

For a rough sense of scale, quantized-model file size follows approximately total_params_billions × bits_per_weight ÷ 8 gigabytes, before accounting for KV cache and context overhead. For a ~35B-parameter class model (even a mixture-of-experts model where only a fraction of those parameters are "active" per token — all experts still have to sit in memory unless you're explicitly offloading unused ones to CPU RAM), the ballpark is:

These are back-of-envelope estimates, not measured figures — actual GGUF file sizes vary by quant recipe and should be checked against the specific file you download. The practical takeaway: on a 3090 Ti, Q4_K_M or a similarly-sized 4-bit quant is usually the realistic ceiling if you want any meaningful context window left over, since the KV cache for a 35B-class model at longer context lengths can add several more gigabytes on top of the weights.

If that math doesn't leave you comfortable, it's worth comparing notes against setups covered elsewhere on SpecPicks — a 6GB VRAM laptop pushed to run Qwen3.6-35B-A3B leans hard on aggressive offloading, while a 32GB M2 Mac field report shows what unified memory buys you at a higher ceiling than the 3090 Ti's dedicated 24GB.

Speculative decoding in llama.cpp: two different paths

llama.cpp supports two conceptually distinct speculative-decoding setups, and it's worth being clear about which one you're using:

1. Classic two-model speculative decoding — a separate, smaller draft model runs alongside the target model, both loaded into VRAM simultaneously, via the project's draft-model flags. This is well-established in llama.cpp but costs extra VRAM for the second model.
2. Self-speculative decoding via an MTP/NextN head — no second model; the target checkpoint's own extra prediction head proposes tokens, which the same model then verifies. This is the path relevant to Qwen3.5/3.6 checkpoints that ship an MTP head, and it's the more VRAM-friendly option on a single 24GB card.

Both approaches share the same underlying idea — propose several tokens speculatively, verify them in one batched pass, keep the ones that match. The difference is just where the "guesser" comes from. For background on why MTP heads exist architecturally and how they're trained, the DeepSeek-V3 repository is the primary reference point the technique traces back to.

Setting up llama.cpp for Qwen3.5/3.6 with MTP

At a high level, the setup looks like this:

1. Build llama.cpp with CUDA support for your installed CUDA toolkit and driver version — use a driver recent enough to match whatever CUDA toolkit version your llama.cpp build targets, and confirm compatibility against NVIDIA's own driver/CUDA compatibility table rather than trusting a specific version number quoted in an unrelated article.
2. Download a GGUF quantization of Qwen3.5 or Qwen3.6 that explicitly documents MTP/NextN head support on its Hugging Face model card — not every quant repack necessarily retains that head, so check before assuming.
3. Confirm your llama.cpp build actually exposes a flag for self-speculative/MTP decoding. Run llama-server --help or llama-cli --help on the exact binary you built and look for the current flag name — this has moved around across llama.cpp releases, so don't rely on a flag name from an older guide without confirming it against your own build's help output.
4. Set context size (--ctx-size) conservatively at first, then increase it while watching nvidia-smi for VRAM headroom — this is the single most common source of out-of-memory crashes on a 24GB card.
5. Benchmark on your own workload using llama.cpp's built-in benchmarking tools rather than assuming a number from an unrelated hardware/model combination will transfer to yours.

What kind of speedup should you actually expect?

The honest answer is: it depends on how often the model's own draft tokens get accepted during verification, which is itself workload-dependent. Structured, predictable output — code completion, JSON, repetitive patterns — tends to have a higher acceptance rate than open-ended creative writing, because the model's own next-token distribution is more confident and predictable in those cases. That means the same Qwen3.5 or Qwen3.6 checkpoint with MTP enabled can show meaningfully different speedups depending on what you're asking it to generate.

Rather than quoting a single tokens-per-second figure here — which would only be true for one specific prompt style, context length, batch size, and llama.cpp build — the more useful exercise is running your own comparison: generate the same prompt with MTP/speculative decoding on and off, and compare wall-clock time and reported tok/s from llama.cpp's own output. If you're coming from a CPU-only setup, SpecPicks' guide to running a 26B model with no GPU at all is a useful baseline for how much a dedicated 24GB card changes the equation regardless of MTP.

Troubleshooting common issues

• CUDA out-of-memory errors: reduce --ctx-size first, then consider a smaller quant if that's not enough headroom. KV cache scales with context length and can be the difference between fitting and not fitting on a 24GB card.
• Speedup is smaller than expected, or negative: this usually means a low draft-acceptance rate for your specific prompt/workload, or that the overhead of the verification pass is eating into the benefit at your current batch size. Try a more structured/predictable prompt style as a sanity check.
• Model loads but MTP doesn't appear to be active: double-check that the GGUF you downloaded actually includes the MTP/NextN tensors (not every repack does) and that your llama.cpp build's help output lists a relevant flag — mismatched builds are a common silent failure mode.
• Driver/CUDA mismatches: keep your NVIDIA driver reasonably current and verify it against the CUDA toolkit version your llama.cpp build was compiled against, rather than assuming any recent driver will work.

Where the RTX 3090 Ti sits in the local-inference landscape

A 3090 Ti's 24GB puts it in an unusual middle tier: comfortably ahead of VRAM-starved laptop and edge setups, but behind newer 32GB+ consumer cards and well behind datacenter parts. If you're weighing whether to lean on the 3090 Ti at all versus other approaches covered on SpecPicks, it's worth reading how far people have pushed the low end — a Raspberry Pi coding agent running Qwen3.6 and a 5GB-VRAM TTS benchmark across 21 GPUs both illustrate how much smaller a footprint you can get away with for lighter workloads, if MTP-on-a-3090-Ti turns out to be more setup effort than your use case justifies. If you already own a 3090 Ti and want to keep it busy beyond LLM inference, it's also the card behind a DIY NVIDIA 3D Vision emitter build using an RP2040.

Citations and sources

• llama.cpp GitHub repository
• DeepSeek-V3 GitHub repository
• Qwen organization on Hugging Face
• NVIDIA GeForce RTX 3090 Ti product page

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.