In brief — 2026-05-19 · Multi-Token Prediction (MTP) — a speculative decoding technique built directly into Qwen 3.6's weights — is delivering real-world 2x inference speed gains on AMD Strix Halo APUs and dual Radeon 9700 AI Pro GPUs, per a benchmark video circulating on r/LocalLLaMA. Coding workloads see the sharpest gains (79–89% token acceptance), making MTP a near-zero-cost throughput upgrade for AMD-based local-LLM rigs — provided llama.cpp is current.
What happened
A benchmark video published May 18, 2026 on YouTube — surfaced through an r/LocalLLaMA thread that climbed the subreddit's front page the same day — demonstrates Multi-Token Prediction accelerating Qwen 3.6 inference on two AMD platforms: a Strix Halo APU system and a dual Radeon 9700 AI Pro setup. The headline figure is a 2x throughput improvement over standard autoregressive decoding.
MTP is a form of speculative decoding baked into the model checkpoint itself, not bolted on afterward with a separate draft model. Where classical speculative decoding requires a smaller companion model to propose candidate tokens, MTP embeds multiple prediction heads directly in the main model weights. The main model simultaneously proposes and verifies several tokens ahead, collapsing what would be multiple sequential forward passes into a single parallel operation.
The gains are not uniform. Community analysis on r/LocalLLaMA puts token acceptance rates for coding tasks at 79–89% with Qwen 3.6 MTP, versus 62–70% for factual or general-purpose tasks. Structured, repetitive output — function signatures, boilerplate, syntax patterns — is highly predictable and easy for the extra heads to guess. Free-form prose is less predictable, narrowing the advantage.
On the implementation side, llama.cpp shipped MTP support and a fresh round of optimizations tracked in pull request #23269. Users running llama-server enable MTP with two startup flags: --spec-type draft-mtp and --spec-draft-n-max 2. A practical caveat from the thread: enabling MTP at the server level causes non-MTP models (Gemma, most other families) to fail at startup, so operators running multi-model servers must handle the flag conditionally per-model.
Results on Apple Silicon have been mixed. At least one M2 Max 96 GB user reported dropping from roughly 12 tokens per second to 9–10 tokens per second after enabling MTP on a 27B model, suggesting the unified-memory architecture does not currently benefit from the same parallelism gains the AMD configurations deliver.
Why it matters for builders
For anyone shopping AMD silicon specifically for local-LLM workloads, MTP support in llama.cpp substantially changes the value calculation — without requiring new hardware.
Strix Halo systems have drawn interest as high-bandwidth unified-memory platforms capable of running 70B+ parameter models at usable speeds, but raw tokens-per-second on autoregressive decoding has been a sticking point versus discrete GPU rigs. A validated 2x MTP gain on Strix Halo narrows that gap materially, particularly for coding-agent use cases (pair programming, code review, test generation) where MTP acceptance is highest.
The dual Radeon 9700 AI Pro configuration represents the class of discrete ROCm-capable cards AMD has positioned for workstation AI inference. MTP's ability to parallelize generation aligns with that silicon's design goals.
Concrete shopping implication: if you are evaluating an AMD AI PC or a ROCm-capable Radeon card and plan to run coding agents locally, MTP support in your chosen model should now be a first-class criterion alongside VRAM capacity and memory bandwidth. Qwen 3 is currently the most accessible MTP-enabled family in llama.cpp.
The flip side: buyers focused on tool-heavy agentic flows — repeated structured API calls, JSON parsing, branching on tool output — should calibrate expectations. Community analysis suggests tool-call patterns sit closer to the factual acceptance range (62–70%), and at least one r/LocalLLaMA thread explored scenarios where MTP is net negative when structured-output constraints reduce predictability.
Hardware angle
Strix Halo is the most direct beneficiary, representing the current ceiling of integrated AI-PC compute in a laptop or mini-PC form factor. The dual Radeon 9700 AI Pro configuration extends the story to discrete ROCm workstations.
None of the current SpecPicks catalog GPUs map directly to these specific SKUs — the Radeon 9700 AI Pro remains primarily a workstation/OEM channel part at this writing. Builders assembling a local-LLM rig today would look to the broader Radeon RX 9000 series for consumer availability, or to the previous generation (RX 7900 XTX, RX 7900 XT) for ROCm-compatible discrete cards that support llama.cpp's Vulkan and HIP backends. Builders running dual-GPU rigs should also budget for thermal management — quality compound like Maxtor MTP-3207 thermal paste is the kind of small line item that gets forgotten until a card throttles mid-inference.
The key operational requirement for capturing MTP gains on any of these platforms is keeping llama.cpp current — PR #23269 landed recent improvements, and a build from two weeks ago may miss meaningful performance work.
What other coverage is saying
The r/LocalLLaMA community has produced several complementary threads in the same news cycle. One thread specifically investigates why MTP may be net negative for tool-heavy agentic flows, concluding that structured output and constrained-format tasks behave more like factual recall than code generation. A separate thread on llama-server and MTP documents the multi-model server problem, noting the current flag design requires careful per-model configuration to avoid breaking non-MTP checkpoints. The Apple Silicon thread adds a useful cross-platform data point: M2 Max users are not seeing the gains AMD benchmarks show, suggesting the speedup is architecture-dependent rather than universal. Separately, Artificial Analysis published benchmarks for Google's Gemini 3.5 Flash this week — not an MTP story, but a reminder that the broader inference-speed competition is intensifying across both cloud and local hardware.
Sources
- YouTube — MTP 2x Faster Token Generation on AMD Strix Halo & Radeon 9700 AI Pro — Primary benchmark video demonstrating MTP gains on AMD platforms with Qwen 3.6.
- r/LocalLLaMA — Time to update llama.cpp to get MTP improvements — Links llama.cpp PR #23269 and discusses updating builds to capture the latest MTP optimizations.
- r/LocalLLaMA — Llama-server and MTP — Documents the
--spec-type draft-mtpand--spec-draft-n-max 2flags and the multi-model compatibility problem. - r/LocalLLaMA — Why might MTP be net negative for tool heavy agentic flows? — Analysis of acceptance-rate variance across task types.
- r/LocalLLaMA — MTP and Apple Silicon, any benefits? — M2 Max 96 GB results showing potential regression versus AMD gains.
- Artificial Analysis — Gemini 3.5 Flash benchmarks — Adjacent inference-speed coverage; Gemini 3.5 Flash scores 55 on the Intelligence Index at 280+ output tokens/sec.
Filed by the SpecPicks News Desk. We summarize and link — never paywall-bypass.