Strix Halo LLM Benchmarks 2026: What AMD's Ryzen AI Max+ Delivers

As of October 2026, AMD's Ryzen AI Max+ 395 ("Strix Halo") decodes a dense Llama 3.1 70B at Q4_K_M at 4.97 tokens per second. That is a near tie with NVIDIA's GB10, the DGX Spark chip, at 4.71 tok/s, and about a third of an M3 Ultra Mac Studio's 14.08 tok/s on the same llama-bench test and the same weight file, per Jeff Geerling's ai-benchmarks results. Where Strix Halo earns its place is mixture-of-experts (MoE) models: a 128GB system runs OpenAI's gpt-oss-120b at 56.14 tok/s on llama.cpp b11146 in a September 26, 2026 re-check (55.57 tok/s on the original May run) and Qwen3.6-35B-A3B at about 62.6 tok/s in llama.cpp's Vulkan backend, per the community-maintained Strix Halo Guide benchmark log. With multi-token prediction (MTP) speculative decoding in llama-server, the same log records a Qwen3.6-35B-A3B MTP quant averaging 101.15 tok/s. The 128GB Framework Desktop lists at $3,449 as of October 16, 2026, marked out of stock. MoE models read only a few billion active parameters per token, so they run fast. Large dense models are limited by memory bandwidth, and Strix Halo's roughly 256GB/s bus is far below the 800GB/s-class Mac Studio chips. The chip's real advantage is capacity: 120B-class MoE models fit on one mini-PC, and no consumer discrete GPU can hold them.

What changed — September and October 2026

• October 16, 2026: MTP speculative-decoding figures added. A new "Speculative decoding (MTP)" subsection lists llama-server results from the Strix Halo Guide: a Qwen3.6-35B-A3B MTP IQ4_XS-Q8nextn quant averaged 101.15 tok/s with draft-n=2 against a 74.88 tok/s no-MTP baseline, and Gemma 4 26B-A4B QAT with a matched MTP head averaged 110.00 tok/s. kyuz0's toolbox README reports Qwen3.8-Flash-Next at 43.9 tok/s decode with an MTP head on an experimental ROCm 10.0 build, versus 28.61 tok/s without MTP in the guide's re-check.
• October 16, 2026: gpt-oss-120b figure in the opening updated to the guide's September 26 b11146 re-check (56.14 tok/s), alongside the original 55.57 tok/s run.
• October 16, 2026: current Framework Desktop price added. Framework's product page lists the Max+ 395 128GB system at $3,449, the 64GB at $1,959 and the Max 385 32GB at $1,269, all marked out of stock.
• October 2026: a same-test comparison against DGX Spark and Mac Studio. A new cross-platform 70B table, in the "Strix Halo vs. DGX Spark vs. Mac Studio" section below, puts four 128GB-class unified-memory machines on one llama-bench run of the same Llama 3.1 70B Q4_K_M file. The Framework Desktop (Ryzen AI Max+ 395) decoded 4.97 tok/s (issue #21), the Dell Pro Max with GB10 4.71 tok/s (issue #34), an M1 Ultra Mac Studio 9.84 tok/s (issue #48) and an M3 Ultra Mac Studio 14.08 tok/s (issue #42).
• October 2026: NVIDIA GB10 / DGX Spark added. The page did not cover Strix Halo's closest rival before. Per NVIDIA's DGX Spark page, GB10 pairs 128GB of LPDDR5x unified memory with 273GB/s of bandwidth on a 256-bit interface, which is only about 7% more bandwidth than Strix Halo.
• October 2026: prefill gap measured. On the same 70B file, GB10 processed a 512-token prompt at 276.17 tok/s against Strix Halo's 34.21 tok/s, about 8× faster, per issue #34 and issue #21. Decode speeds stay within about 6% of each other.
• October 2026: power draw added. Peak system power during the 70B run was 133W for the Framework Desktop, 156W for the GB10 system and 243W for the M3 Ultra Mac Studio, per the ai-benchmarks results table.
• October 2026: opening rewritten to lead with the dense-70B result from the new table, alongside the existing MoE figures.
• Measured decode figures added. The earlier version framed Strix Halo on memory capacity alone. The refresh adds a table of six community-measured llama.cpp decode figures on a 128GB Ryzen AI Max+ 395, including gpt-oss-120b at 55.57 tok/s and Qwen3.6-35B-A3B at 62.56 tok/s on Vulkan, per the Strix Halo Guide benchmark log. Each row also lists its Hugging Face weight size and a 256GB/s bandwidth ceiling.
• The "70B fits" claim is now qualified with a speed. A dense Llama 3.1 70B at Q4_K_M fits in memory but decodes at only 4.7–4.9 tok/s, under its ~6 tok/s bandwidth ceiling, per the same Strix Halo Guide log.
• The ROCm section was rewritten. The paragraph saying the ROCm gap "narrowed through 2025" is gone. In its place are measured results: Vulkan led decode at 97.73 vs 73.65 tok/s on Qwen3-Coder-30B-A3B, and ROCm led pp512 prefill at 1,344.65 vs 1,115.30 tok/s, per Soothill's August 2026 comparison. The section also notes kyuz0's stable ROCm 10.0 gfx1151 toolbox.
• Links updated. The llama.cpp link now points to the current ggml-org/llama.cpp repository. A bare r/LocalLLaMA link that carried no figure was removed.

What "Strix Halo" Actually Is (and Isn't)

"Strix Halo" is AMD's codename for the Ryzen AI Max 300 series. It is a mobile/desktop-class APU that pairs Zen 5 CPU cores with an RDNA 3.5 integrated GPU (the Radeon 8060S on the top part, LLVM target gfx1151) and a unified LPDDR5X memory subsystem. AMD announced it at CES 2025. It ships in mini-PCs and high-end laptops such as the Framework Desktop, GMKtec EVO-X2, Beelink GTR9 Pro, and ASUS ROG Flow Z13. It is not in the same product tier as AMD's Instinct MI300X or Radeon Pro W7900. Those are datacenter and workstation-class discrete GPUs, sold separately from any host CPU and built for server-scale training and inference with their own HBM or GDDR memory. Coverage that mixes Strix Halo throughput figures with MI300X or W7900 numbers compares two different product categories, so this piece keeps them separate. For how the same chip handles gaming workloads, see SpecPicks' Strix Halo Gaming Benchmarks: What AMD's APU Can Really Do.

Why Strix Halo Matters for Local LLM Inference

For local LLM use, the key spec is memory capacity, not raw compute. Per AMD's published specifications, the top Ryzen AI Max+ 395 configuration has a 16-core/32-thread Zen 5 CPU, a 40-compute-unit RDNA 3.5 iGPU, and up to 128GB of on-package LPDDR5X-8000 memory on a 256-bit bus. The OS typically reserves about 2GB, which leaves well over 120GB as a shared pool for the GPU, NPU, and CPU. A discrete consumer GPU like the RTX 4090 tops out at 24GB of VRAM. That forces heavy quantization or model-splitting for anything much past a 30B-parameter model. A Strix Halo system can give the GPU a far larger share of memory. It can hold 70B dense weights and 100B-plus MoE weights entirely in memory, with no offloading to system RAM or disk. SpecPicks' Strix Halo vs. RTX 3060 12GB: Unified Memory or Discrete VRAM? and Best Mini PC for Local LLMs in 2026 cover that capacity-versus-bandwidth tradeoff in more depth.

What's Changed as of September 2026: Measured Strix Halo Numbers

The table below collects public community measurements of current open-weight models on a 128GB Ryzen AI Max+ 395. All rows are llama.cpp llama-bench token-generation (tg128) figures with no speculative decoding. File sizes are the summed weight files for the exact quant, taken from each Hugging Face repository.

The "bandwidth ceiling" column is a physics sanity check: 256GB/s divided by the bytes read per token. For MoE models, that is the file size scaled by active ÷ total parameters. Real decode speed always lands below this ceiling, and every row here does.

How to read it:

• Active parameters, not total size, set decode speed. The 22GB Qwen3.6-35B-A3B runs about 13× faster than the 42GB dense Llama 3.1 70B because it reads roughly 2GB of weights per token instead of 42GB.
• 120B-class models are practical, not fast. gpt-oss-120b is the standout because only 5.1B parameters are active per token. Per the Strix Halo Guide, its prompt processing also held at 293.73 tok/s even at a 65,536-token prompt. Nemotron 3 Super and Qwen3.5-122B, with 10–12B active parameters, land around 18–23 tok/s: usable for chat, slow for agent loops.
• The largest models barely fit. The 93.7GB Qwen3.8-Flash-Next file loads within the 128GB pool, but the guide notes that its September 26 re-check ran last and filled swap. Treat models near 90GB+ as tight fits that leave little room for long-context KV cache. MTP changes the picture for this model: kyuz0's experimental rocm-10.0-strix-llama toolbox lists Qwen3.8-Flash-Next Q4_K_XL with Unsloth's shared Q8 MTP head at 43.9 tok/s decode and 1,207 tok/s pp2048, per the toolbox README. That is a different quant, backend and decoding method from the table row, so the two figures are not a like-for-like comparison.
• Numbers move with the software stack. The guide's September 26 re-check on llama.cpp b11146 raised prompt processing 13–28% over the same-night original builds, while decode moved only a few percent. Two boxes on different kernels, Mesa versions, or llama.cpp builds can disagree by a few tok/s. The guide's README treats about 2% tg128 spread as normal between well-matched systems.

Throughput also falls as context fills: per the same guide, Qwen3.6-35B-A3B decodes at 32.23 tok/s after a filled 128K KV cache, versus about 62 tok/s at short context.

Speculative decoding (MTP)

Multi-token prediction drafts several tokens per step and verifies them in one pass, so it raises effective decode speed above the plain tg128 figures in the table. kyuz0's toolbox README notes that MTP support has been merged into mainline llama.cpp and is available in the standard toolboxes. The Strix Halo Guide measured these llama-server averages over six prompts on Vulkan RADV:

The guide cautions against reading these as direct llama-bench results or as a general claim that local models run at 100+ tok/s. In a separate Ollama check, it found that MTP output at temperature 0 differed from no-draft output, so output equality is not established. MTP roughly doubles the dense 27B route, but that route stays far slower than the MoE models.

The Real Bottleneck: Memory Bandwidth, Not Compute

Community benchmark logs such as the Strix Halo Guide and discussion on the llama.cpp GitHub repository consistently identify memory bandwidth, not raw TFLOPs, as the binding limit on Strix Halo's token-generation ("decode") speed. Autoregressive decoding is largely memory-bandwidth-bound, because each new token requires re-reading the active weights from memory. AMD's published ~256GB/s figure for the 256-bit LPDDR5X-8000 bus is far below the roughly 1TB/s of an RTX 4090's GDDR6X and the multi-terabyte-per-second HBM3 of datacenter parts like the MI300X. The dense 70B row above shows the result: 42.5GB of weights over a 256GB/s bus caps decode near 6 tok/s before any software overhead, and the measured 4.7–4.9 tok/s sits just under that cap. Strix Halo gives up raw decode speed in exchange for running models that won't fit on a VRAM-limited discrete card at all. That is a tradeoff, not an unqualified win. Prompt processing (prefill) is more compute-bound and scales better: the guide's gpt-oss-120b run measured 726.99 tok/s pp512 against 55.57 tok/s decode.

Software Maturity: ROCm vs. Vulkan

The early Strix Halo LLM community leaned on llama.cpp's Vulkan backend because AMD's ROCm stack had historically supported consumer RDNA hardware less well than Instinct-class GPUs. ROCm has since shipped native gfx1151 support. kyuz0's widely used Strix Halo llama.cpp toolboxes now list a stable ROCm 10.0 container alongside the Vulkan RADV one. The README still describes Vulkan RADV as the most stable and compatible choice for most users, and its setup steps label the ROCm option "Recommended for Performance."

The measured picture as of 2026 is a split by workload, not a single winner:

• Vulkan usually wins decode. In Soothill's August 2026 comparison, Qwen3-Coder-30B-A3B Q4_K_S generated 97.73 tok/s on Vulkan versus 73.65 tok/s on ROCm.
• ROCm usually wins prompt processing. The same comparison measured ROCm ahead on pp512, at 1,344.65 versus 1,115.30 tok/s for Qwen3-Coder and 339.9 versus 285.0 tok/s for Qwen3.5-122B-A10B. The Strix Halo Guide found the same pattern: HIP was 24.8% faster at pp16384 on Qwen3.6-35B-A3B, while Vulkan was 18.1% faster at tg128.

In practice, use Vulkan for interactive chat and coding. Test ROCm for RAG ingestion, long-document summarization, and batch serving, where prefill dominates. Either way, verify current driver and backend support for your specific model and quantization before buying rather than assuming parity with Nvidia's CUDA ecosystem. kyuz0's toolbox notes, for example, still recommend flash attention and --no-mmap on Strix Halo to avoid crashes and slowdowns. SpecPicks' Ryzen AI Halo mini-PC coverage and DGX Spark rival review roundup track this in more detail.

Strix Halo vs. Discrete GPU Options

Figures come from each vendor's published product specifications, not independent testing. Sustained throughput depends heavily on software stack, quantization, and workload. A model that fits in 24–32GB of VRAM, such as the 22GB Qwen3.6-35B-A3B quant, has a far higher bandwidth ceiling on an RTX 4090 or 5090; the 63GB gpt-oss-120b doesn't fit on either card without offloading. For discrete-GPU context in the same class of AI workload, see RTX 5090 Benchmark 2026: 4K Gaming & AI vs. AMD and RTX 5090 AI Desktop: 2026 Build Guide & Benchmarks.

Strix Halo vs. DGX Spark vs. Mac Studio: One 70B Test, Four Unified-Memory Machines

Vendor bandwidth specs explain the theory. A single benchmark series run the same way on several machines shows the practice. Jeff Geerling's ai-benchmarks project ran llama.cpp's llama-bench against the same file, bartowski's Meta-Llama-3.1-70B-Instruct Q4_K_M (39.59 GiB, about 42.5GB), with every layer on the GPU. Each cell below links the run it came from.

Every decode figure sits under its bandwidth ceiling, which is bandwidth divided by the roughly 42.5GB read per token: about 6 tok/s for Strix Halo, about 6.4 tok/s for GB10, and about 19 tok/s for the two 800GB/s-class Mac Studios.

How to read it:

• Strix Halo and DGX Spark decode at the same speed. Their buses are within about 7% of each other (256 vs. 273GB/s), and their 70B decode rates are within about 6% (4.97 vs. 4.71 tok/s). For chat with a dense 70B model, neither one is meaningfully faster.
• GB10 wins prefill by a wide margin. It processed a 512-token prompt about 8× faster (276.17 vs. 34.21 tok/s), and the gap held at 4,096 tokens (223.82 vs. 32.54 tok/s). Prefill is compute-bound, so this is where NVIDIA's Blackwell tensor cores and CUDA stack show. The Strix Halo run used an August 2025 pre-production unit on Vulkan, so a current ROCm build may narrow the gap. The Soothill comparison above shows ROCm ahead on prefill. No public same-file 70B re-run on current software was found for this update.
• An older Mac Studio still doubles Strix Halo on dense 70B. The 2022 M1 Ultra decoded 9.84 tok/s, about 2× Strix Halo, because its 800GB/s bus is about 3× wider. The M3 Ultra reached 14.08 tok/s, about 2.8× Strix Halo. It also drew the most power of the three machines with a reported figure (243W vs. 133W).
• Strix Halo ran at the lowest reported power. At 133W peak, the Framework Desktop drew about 15% less than the GB10 system's 156W for slightly faster decode on this test.

For buyers whose workload is dense 70B chat, this table argues for 800GB/s-class Apple silicon over either 128GB x86 or ARM box. For long-prompt work, GB10 has the edge. For MoE models up to about 120B parameters, Strix Halo's measured 55.6 tok/s on gpt-oss-120b (Strix Halo Guide) is the practical case for the platform.

Building an AMD-Based AI Rig vs. Buying a Strix Halo Mini-PC

The unified-memory architecture is currently exclusive to the mobile/mini-PC Ryzen AI Max lineup. As of October 16, 2026, Framework's product page lists the Framework Desktop at $3,449 for the Max+ 395 with 128GB, $1,959 for the Max+ 395 with 64GB and $1,269 for the Max 385 with 32GB, with all three configurations marked out of stock. Readers considering a DIY AMD desktop instead should know that a standard AM4/AM5 build still pairs a conventional motherboard, such as the ASUS ROG Strix B550-F Gaming Motherboard ($252.19), with a separate discrete GPU rather than an integrated unified-memory chip. Fast local storage matters on either platform. Weight files for the models above run from 22GB to over 90GB and must be read from disk at startup and during model swaps. An enclosure like the ASUS ROG Strix Arion USB 3.2 M.2 NVMe Enclosure ($62.28) speeds that up for external or swappable storage. Long inference sessions also generate meaningful heat, so case airflow is a reasonable line item for a dedicated AI-rig build. The Cooler Master MasterFan MF120 Halo 3-in-1 ARGB fans ($108.96) are one option. Prices reflect the listing at time of publication and may vary, so check the current price before purchasing.

Cost-Performance Takeaways

• Strix Halo's advantage is capacity per dollar for large quantized models that don't fit in 16–32GB of discrete VRAM, not raw tokens per second.
• Pick MoE models: per the community figures above, active-parameter count, not total size, decides decode speed.
• Discrete GPUs with less memory but far more bandwidth (RTX 4090, RTX 5090) still generally win on decode speed for models that fit in their VRAM.
• Datacenter parts like the MI300X aren't a relevant price/performance comparison for a home or small-business local-LLM rig. They are server accelerators, not desktop or mini-PC components.
• Backend choice matters: Vulkan for generation-heavy use and ROCm for prompt-heavy use, per the public comparisons above. Verify current support for your target model and quantization before buying.

Bottom Line

"Strix Halo benchmark LLM" searches often turn up a mix of consumer-APU and datacenter-GPU numbers that don't belong in the same comparison. Judged on its own terms, the Ryzen AI Max+ 395's pitch for local LLM work is unified memory capacity rather than raw throughput. As of October 2026, public community benchmarks show it running 120B-class MoE models such as gpt-oss-120b at about 56 tok/s, a class of model no single consumer discrete card can hold. Dense 70B models remain bandwidth-bound at under 5 tok/s. ROCm now has a stable gfx1151 release, but Vulkan remains the faster decode path in public comparisons. For broader context on how fast the open-weight model landscape is moving around hardware like this, see Open-Weight Models Caught Up on Cyber Benchmarks in Four Months.

Citations and sources

• https://github.com/hogeheer499-commits/strix-halo-guide/blob/main/BENCHMARKS.md
• https://github.com/hogeheer499-commits/strix-halo-guide
• https://www.soothill.io/blog/2026/08/03/llamacpp-vulkan-vs-rocm-strix-halo/
• https://github.com/kyuz0/amd-strix-halo-toolboxes
• https://github.com/ggml-org/llama.cpp
• https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
• https://huggingface.co/ggml-org/gpt-oss-120b-GGUF
• https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
• https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF
• https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF
• https://huggingface.co/bartowski/Meta-Llama-3.1-70B-Instruct-GGUF
• https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html
• https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
• https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
• https://frame.work/products/desktop-diy-amd-aimax300
• https://github.com/geerlingguy/ai-benchmarks
• https://github.com/geerlingguy/ai-benchmarks/issues/21
• https://github.com/geerlingguy/ai-benchmarks/issues/34
• https://github.com/geerlingguy/ai-benchmarks/issues/42
• https://github.com/geerlingguy/ai-benchmarks/issues/48
• https://www.nvidia.com/en-us/products/workstations/dgx-spark/
• https://www.apple.com/newsroom/2022/03/apple-unveils-m1-ultra-the-worlds-most-powerful-chip-for-a-personal-computer/
• https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac-studio-the-most-powerful-mac-ever/

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.