Skip to main content

Strix Halo Clustering for Local LLMs: What the LocalLLaMA Reports Show

Synthesizing LocalLLaMA community measurements on AMD's 128 GB unified-memory APU as a single-box and clustered LLM host.

Per See Hiong's August 2026 llama.cpp b10440 run on a GMKtec EVO-X2, one 128 GB Strix Halo box decodes gpt-oss-120b at 49 tok/s. A May 2026 three-node USB4 test shows clustering adds memory, not speed. Buy one unless the model won't fit.

Where this fits: the 128 GB unified tier of the local-LLM GPU picks by VRAM (one card per tier, with cited tok/s). Tiers: 8 GB · 12 GB · 16 GB · 24 GB · 48 GB+ · 128 GB unified.

Strix Halo Clustering for Local LLMs: What the LocalLLaMA Reports Show

Direct answer

One 128 GB Ryzen AI Max+ 395 (Strix Halo) box decodes gpt-oss-120b at 49.0 tok/s on llama.cpp b10440, per an August 2026 run on a GMKtec EVO-X2. For a same-tester comparison with NVIDIA's DGX Spark, an October–November 2025 llama-bench thread measured an EVO-X2 at 47.49 tok/s against 60.57 tok/s for the Spark on the same model and test (source). At a 32,768-token context the two are closer, at 35.39 vs 40.55 tok/s, but the Spark still prefills 2.95× faster (1,027 vs 349 tok/s). Clustering Strix Halo boxes adds memory, not speed. A May 2026 three-node USB4 matrix ran MiniMax-M2.7, which needs more than one box, at 21.4 tok/s on two nodes and 19.7 tok/s on three. The links in published Strix Halo clusters run at 5 to 10 Gbps. NVIDIA instead ships each DGX Spark with a 200 Gbps ConnectX-7 NIC and supports linking up to four units. If the model fits in 128 GB, buy one box. Buy a second only when the model will not load on one.

What changed since February 2026: a single Strix Halo box now decodes gpt-oss-120b at 49.0 tok/s on llama.cpp b10440 (See Hiong, August 2026), and a May 2026 three-node USB4 llama-bench matrix measured MiniMax-M2.7 (140.8 GB, too large for one box) at 21.4 tok/s on two nodes and 19.7 tok/s on three (strix-halo-guide issue #12), so adding nodes past the minimum that fits the model costs speed.

What changed — October 2026

  • New Strix Halo vs DGX Spark table (October 23 update). It compares the two boxes on the same gpt-oss-120b MXFP4 llama-bench test at five context depths, from 0 to 32,768 tokens. The Strix Halo figures are from a GMKtec EVO-X2 on ROCm; the DGX Spark figures are from the same tester's CUDA run. At zero depth, decode is 47.49 vs 60.57 tok/s and prefill is 999.59 vs 1,956.03 tok/s.
  • Added the NVIDIA DGX Spark as the main alternative. It has 128 GB of LPDDR5x at 273 GB/s and a ConnectX-7 NIC at 200 Gbps, per NVIDIA, against Strix Halo's 256 GB/s (AMD). On llama.cpp's Llama 2 7B Q4_0 scoreboards, with no flash attention, it decodes at 57.21 tok/s on CUDA (CUDA scoreboard) against 50.01 tok/s for the Ryzen AI Max+ 395 on ROCm (ROCm scoreboard).
  • Prices re-checked October 23, 2026. The GMKtec EVO-X2 128 GB still lists at $3,499.99 in SpecPicks' tracked Amazon listings. The ACEMAGIC M1A PRO+ listing currently shows no price.
  • Newer single-node and multi-node measurements (October 17 refresh). An August 16, 2026 write-up ran gpt-oss-120b MXFP4 on a GMKtec EVO-X2 at 49.0 tok/s decode and 165.9 tok/s prompt processing (llama.cpp b10440, ROCm 7.x, 8,192-token context, Flash Attention on), well above the 33.12 tok/s single-node figure from Geerling's August 2025 test. A May 2026 community RPC matrix on three Corsair AI Workstation 300 boxes over a USB4 mesh measured MiniMax-M2.7 UD-Q4_K_XL at 21.4 tok/s on two nodes and 19.7 tok/s on three (ROCm 7.2), and Qwen3-Coder-Next UD-Q8_K_XL at 40.1 tok/s on one node, 34.5 on two and 29.4 on three (Vulkan/RADV). Rows added to the table below.
  • Current-generation model rows with cited tok/s. The earlier version cited no source for any throughput figure. A new table now lists Llama 3.1 70B, Qwen3-Coder-Next, MiniMax M2.5, gpt-oss-120b, Qwen3.5-397B-A17B and Kimi K2.5, each with a measured decode rate and a bandwidth-ceiling check against its Hugging Face weight-file size. Sources: strix-halo-llm-perf, AMD's cluster guide, Jeff Geerling.
  • Cluster scaling claim corrected. The earlier version said two nodes deliver about 1.2× single-node generation and four nodes 1.4× to 1.6×. The published measurements show the opposite for models that fit on one node. Qwen3-Coder-Next at Q6_K_XL decoded at about 38.7 tok/s on one host and about 26.3 tok/s over two-host RPC, per strix-halo-llm-perf. gpt-oss-120b dropped from 33.12 to 24 tok/s across four nodes, per Jeff Geerling.
  • Interconnect speed corrected. The earlier version assumed 40 Gbps USB4. The strix-halo-llm-perf hosts measured about 9.4 Gbps over a direct USB4/Thunderbolt cable. Geerling "could only get 10 Gbps" over Thunderbolt/USB4 and used the built-in 5 Gbps NICs.
  • Prefill time corrected. The earlier "8 to 14 seconds" for a 4K-token 70B prompt was too optimistic. At the measured 81.6 tok/s pp512 rate for Llama 3.1 70B Q4_K_M (strix-halo-llm-perf), a 4,096-token prompt takes about 50 seconds. AMD's four-node Kimi K2.5 cluster reported a 39.7 s time to first token at 4,096 tokens with Flash Attention on.
  • Prices updated. The earlier version quoted $1,800 to $2,400 per system. The 128 GB Strix Halo mini PCs SpecPicks tracks on Amazon now list at $3,299.00 (ACEMAGIC M1A PRO+, October 12, 2026) and $3,499.99 (GMKtec EVO-X2, October 13, 2026).
  • Mac comparison updated, spec link fixed. Apple's Mac Studio spec page now lists the M5 Max, with up to 128 GB at 614 GB/s, and the M5 Ultra, with up to 512 GB at 1.2 TB/s. The earlier M4 Max range of "11 to 16 tok/s" on 70B Q4 had no source, and its top end exceeded the ceiling implied by the 546 GB/s the article itself listed (546 / 42.5 GB = 12.8 tok/s). The AMD product link, which redirected to amd.com's homepage, now points to the Ryzen AI Max+ 395 spec page.

Current-generation model rows (October 2026)

Decode on Strix Halo is bounded by memory bandwidth. Per AMD's spec page, the chip uses 256-bit LPDDR5x-8000, which works out to 256 GB/s. A dense model cannot decode faster than 256 GB/s divided by its weight-file size. A mixture-of-experts (MoE) model reads only its active experts per token, so its ceiling is 256 GB/s divided by the active share of the file. Every row below sits under that ceiling. File sizes are the summed Hugging Face GGUF weight files for the quant named.

Model (type)Quant, HF file sizeNodes / linkDecode tok/sPrefill tok/sBandwidth ceiling checkSource
Llama 3.1 70B (dense)Q4_K_M, 42.5 GB15.181.6 (pp512)256 / 42.5 = 6.0strix-halo-llm-perf
Qwen3-Coder-Next (MoE, 80B / 3B active)Q4_K_M, 48.5 GB142.7531 (pp512)~1.8 GB active → ~140strix-halo-llm-perf
MiniMax M2.5 (MoE, 229B, 8 of 256 experts active)Q3_K_M, 109.3 GB132.8156 (pp512)~4.8 GB active (~10B) → ~53strix-halo-llm-perf
gpt-oss-120b (MoE, 117B / 5.1B active)MXFP4, 63.4 GB133.12216.93 (pp512)~2.8 GB active → ~93Jeff Geerling
gpt-oss-120b (same model, clustered)MXFP4, 63.4 GB424not reportedslower than one nodeJeff Geerling
gpt-oss-120b (same model, Aug 2026 build b10440, ROCm 7.x)MXFP4, 63.4 GB149.0165.9 (llama-cli, 8,192 ctx)~2.8 GB active → ~93See Hiong
Qwen3-Coder-Next (MoE, 80B / 3B active)UD-Q8_K_XL, 86.3 GB1 / 2 / 3, USB4 mesh (Vulkan/RADV)40.1 / 34.5 / 29.4709.8 / 690.5 / 679.9 (pp512)each added node is slowerstrix-halo-guide #12
MiniMax-M2.7 (MoE, 230B / 10B active)UD-Q4_K_XL, 140.8 GB2 / 3, USB4 mesh (ROCm 7.2)21.4 / 19.7238.6 / 236.3 (pp512)~6.1 GB active → ~42; does not load on 1 nodestrix-halo-guide #12
Qwen3.5-397B-A17B (MoE, 17B active)UD-Q4_K_XL, 245.3 GB2, USB4 (~9.4 Gbps)11.76147.55 (pp512)~10.5 GB active → ~24strix-halo-llm-perf
Kimi K2.5 (MoE, 1T / 32B active)UD-Q2_K_XL, 374.8 GB4, 5 Gbps Ethernet9.45 (8.30 at 8,192 seq)100.77 (batch 4096)~12 GB active → ~21AMD

Two patterns stand out. The only dense 70B row decodes at 5.1 tok/s, close to its 6.0 tok/s ceiling, so no software update will make dense 70B models fast on this chip. Sparse MoE models are the reason Strix Halo became interesting: MiniMax M2.5, with roughly 230B parameters, decodes about six times faster than dense Llama 3.1 70B on the same box because each token touches only about 10B parameters. The active-parameter count for MiniMax comes from the 8-of-256-expert routing in its config.json and the 10B-active figure MiniMax publishes for the M2 architecture. gpt-oss-120b's 5.1B active count is from OpenAI's model card, and Qwen3-Coder-Next's 3B active count is from Qwen's model card.


Strix Halo vs DGX Spark: one model, one benchmark (October 2026)

Most people who consider a Strix Halo cluster also look at NVIDIA's DGX Spark, which pairs 128 GB of unified memory with a ConnectX-7 NIC. Comparisons between the two usually mix different runtimes, models and context lengths. The table below avoids that. Every row is one tester's llama-bench run of gpt-oss-120b MXFP4 (59.02 GiB) with flash attention on and a 2,048-token micro-batch, measured at five context depths. The Strix Halo box is a GMKtec EVO-X2 on ROCm (HIP), llama.cpp build 03792ad (source). The DGX Spark ran CUDA on llama.cpp build 7db35a7, after a kernel fix the same tester found (source). Decode is tg32 and prefill is pp2048.

Context depth (tokens)Strix Halo decode (tok/s)DGX Spark decode (tok/s)Strix Halo prefill (tok/s)DGX Spark prefill (tok/s)
047.49 (EVO-X2)60.57 (DGX Spark)999.59 (EVO-X2)1,956.03 (DGX Spark)
4,09644.23 (EVO-X2)54.14 (DGX Spark)824.37 (EVO-X2)1,637.34 (DGX Spark)
8,19242.52 (EVO-X2)51.54 (DGX Spark)703.42 (EVO-X2)1,512.01 (DGX Spark)
16,38439.71 (EVO-X2)47.45 (DGX Spark)514.89 (EVO-X2)1,307.42 (DGX Spark)
32,76835.39 (EVO-X2)40.55 (DGX Spark)348.59 (EVO-X2)1,027.31 (DGX Spark)

How to read the table:

  • Decode is close, because both boxes are limited by memory bandwidth. The DGX Spark has 273 GB/s on a 256-bit bus (NVIDIA). Strix Halo has 256 GB/s (AMD). The Spark's decode lead is 28% at zero depth (60.57 vs 47.49 tok/s) and 15% at 32K (40.55 vs 35.39). On llama.cpp's Llama 2 7B Q4_0 scoreboards, with no flash attention, the gap is 14%: 57.21 tok/s on the DGX Spark (CUDA scoreboard) and 50.01 tok/s on the Ryzen AI Max+ 395 (ROCm scoreboard).
  • Prefill is where the DGX Spark pulls away, and the gap grows with context. It processes prompts 1.96× faster at zero depth and 2.95× faster at 32K. For a 32K-token agent or RAG prompt, prefill at those rates takes about 32 seconds on the Spark and about 94 seconds on Strix Halo. Those are rough figures: 32,768 tokens divided by each row's pp2048 rate.
  • One kernel setting changed the DGX Spark result by about a third. Before the fix, the same tester measured the Spark at 45.87 tok/s decode, below the Strix Halo box's 47.49 (source). Treat either platform's numbers as tied to a specific kernel and build.
  • Clustering is where the two platforms differ most. Published Strix Halo clusters link nodes at 5 to 10 Gbps over Ethernet or USB4, and every added node slows decode. A third node took MiniMax-M2.7 from 21.4 to 19.7 tok/s (strix-halo-guide #12). The DGX Spark ships with a 200 Gbps ConnectX-7 NIC, and NVIDIA says up to four units can work together on larger models (NVIDIA). No public same-model, multi-node llama.cpp comparison of the two platforms was found, so whether that faster link changes the scaling result is unmeasured.

The GMKtec EVO-X2 128 GB used in the Strix Halo run lists at $3,499.99 in SpecPicks' tracked Amazon listings on October 23, 2026. SpecPicks does not track a DGX Spark listing, so this comparison makes no price-per-token claim for it.

Why unified-memory APUs matter for LLM hosts

The discrete-GPU model of LLM inference assumes you have enough VRAM to hold the entire model plus KV cache, and that anything which does not fit must cross the PCIe bus. For models up to ~30B parameters quantized to 4 bits, a single RTX 4090 (24 GB) or RTX 5090 (32 GB) handles the workload comfortably. For 70B+ models, you either accept aggressive CPU offload (which collapses tok/s) or you stack two GPUs across PCIe.

Unified-memory APUs dissolve that boundary. Per AMD's spec page, the Ryzen AI Max+ 395 supports up to 128 GB of 256-bit LPDDR5x-8000 (256 GB/s), shared between the CPU cores, the 40-CU Radeon 8060S integrated GPU and the NPU. For a 70B model whose Q4_K_M file is 42.5 GB, plus KV cache for long contexts, Strix Halo holds the whole model in working memory without CPU offload.

That capability has driven a wave of community experimentation on r/LocalLLaMA. The unified memory llm thread series throughout 2025 and early 2026 has documented benchmark numbers, quantization choices, and multi-node clustering configurations using Strix Halo mini-PCs and prototype dev boards. Synthesizing those reports against published spec sheets and against the more familiar discrete-GPU baselines is the goal of this article.

The amd strix halo llm question is not "is Strix Halo faster than a 4090?" (it is not for models that fit in 24 GB). The question is "is Strix Halo a better dollar-per-token-for-large-models option than chaining two 4090s or buying a 128 GB Mac Studio?" That comparison is where the picture gets interesting.

Key Takeaways

  • A single 128 GB Strix Halo box decodes dense Llama 3.1 70B Q4_K_M at 5.1 tok/s, close to the 6.0 tok/s bandwidth ceiling, per strix-halo-llm-perf.
  • Sparse MoE models are the platform's strength: MiniMax M2.5 (Q3_K_M) decodes at 32.8 tok/s and Qwen3-Coder-Next (Q4_K_M) at 42.7 tok/s on one box, per the same repo.
  • Clustering buys capacity, not speed. Two nodes run Qwen3.5-397B-A17B at 11.76 tok/s (strix-halo-llm-perf) and four run the 1T-parameter Kimi K2.5 at 9.45 tok/s (AMD), but a model that fits on one node decodes slower when split (gpt-oss-120b: 33.12 → 24 tok/s across four nodes, per Jeff Geerling). A third node slows even a model that needs sharding: MiniMax-M2.7 ran at 21.4 tok/s on two nodes and 19.7 tok/s on three in a May 2026 USB4 matrix.
  • Real-world node links run at about 5 to 10 Gbps, not USB4's 40 Gbps headline, per both cluster write-ups.
  • 128 GB systems now list at $3,299.00 to $3,499.99 in SpecPicks' tracked Amazon listings (October 12–13, 2026), so a four-node cluster costs roughly $13,000 to $14,000 before networking.

Why is Strix Halo interesting for LLM inference?

Strix Halo's interest for the LLM community comes from three converging properties. First, the integrated 40-CU Radeon 8060S GPU has direct access to up to 128 GB of memory at 256 GB/s (AMD), a capacity no consumer discrete GPU offers. Second, the platform is x86, which means the entire llama.cpp, vLLM, and Ollama software stack runs on it without porting effort. Third, llama.cpp runs on it through both ROCm (HIP) and Vulkan backends; AMD's 2026 cluster guide builds llama.cpp against ROCm 7 with rocWMMA Flash Attention for the gfx1151 target, and strix-halo-llm-perf found Vulkan often decodes faster while ROCm often prefills faster.

The amd strix halo llm community immediately recognized that the bottleneck for large-model inference at home is not raw compute, it is memory capacity and bandwidth. The RTX 4090 pairs 24 GB of GDDR6X with a 384-bit bus (NVIDIA), a much wider memory interface than Strix Halo's 256-bit LPDDR5x but a fraction of its capacity. Strix Halo has 256 GB/s but 128 GB. For a memory-bandwidth-bound workload like LLM token generation, the 4090 wins per-token throughput. For workloads where the model does not fit on the 4090 and you would otherwise need to offload layers to system RAM over PCIe, Strix Halo's unified pool is faster end-to-end.

That trade-off makes Strix Halo a natural fit for large MoE models where the 4090 needs aggressive offload. Apple's alternative is a Mac Studio with the M5 Max (up to 128 GB at 614 GB/s) or M5 Ultra (up to 512 GB at 1.2 TB/s), per Apple's spec page.

What does the LocalLLaMA Strix Halo cluster setup look like?

Published Strix Halo clusters use two to four mini PCs or Framework Desktop boards running llama.cpp's RPC backend, which splits a model's layers across machines. AMD's February 2026 guide used four Framework Desktops (Ryzen AI Max+ 395, 128 GB each) on Ubuntu 24.04.3 with ROCm, linked over 5 Gbps Ethernet. strix-halo-llm-perf paired a GMKtec EVO-X2 and a Beelink GTR9 Pro over a direct USB4/Thunderbolt cable. Jeff Geerling clustered four Framework mainboards and tested llama.cpp RPC, Exo and distributed-llama.

Layer splitting is pipeline parallelism: each token still passes through every layer in sequence, so a second node adds memory, not decode speed. Geerling summed it up: inference "is always going to be slower across multiple machines, than if you can just run the model on one machine." He also reported llama.cpp RPC segfaulting on DeepSeek R1 at Q4_K_M, and running DeepSeek R1 671B at Q2_K across the four nodes at 8.25 tok/s decode.

The interconnect is the binding constraint, and it is slower than the spec sheet suggests. strix-halo-llm-perf measured about 9.4 Gbps over USB4/Thunderbolt. Geerling "could only get 10 Gbps" over Thunderbolt/USB4 and reached the built-in NICs' 5 Gbps without trouble. AMD's guide used the 5 Gbps Ethernet ports.

Spec table: Strix Halo vs RTX 4090 vs Mac Studio M5 Max

SpecStrix Halo (Ryzen AI Max+ 395)RTX 4090 (single GPU)Mac Studio M5 Max (40-core GPU)
Memory capacityUp to 128 GB unified (AMD)24 GB GDDR6X (NVIDIA)Up to 128 GB unified (Apple)
Memory bandwidth256 GB/s (256-bit LPDDR5x-8000, AMD)384-bit bus (NVIDIA)614 GB/s (Apple)
GPURadeon 8060S, 40 CUs (AMD)Ada Lovelace (NVIDIA)40-core GPU (Apple)
Board powervaries by system450 W total graphics power (NVIDIA)not published as a figure
System price (Oct 2026)$3,299.00–$3,499.99 for 128 GB mini PCs (SpecPicks' tracked Amazon listings, Oct 12–13, 2026)varies by buildsee Apple's configurator
Software stackROCm or Vulkan + llama.cppCUDAMetal + llama.cpp / MLX
Llama 3.1 70B Q4_K_M (42.5 GB)Fits in memoryDoes not fit; needs offloadFits in memory

Quantization matrix for a dense 70B model: file size + bandwidth ceiling

File sizes are the bartowski Llama 3.1 70B Instruct GGUF weight files. The ceiling is Strix Halo's 256 GB/s (AMD) divided by file size; real decode lands somewhat below it once KV-cache reads are counted.

QuantizationLlama 3.1 70B file sizeStrix Halo decode ceilingMeasured on Strix Halo
Q3_K_M34.3 GB7.5 tok/snot reported
Q4_K_M42.5 GB6.0 tok/s5.1 tok/s (strix-halo-llm-perf)
Q5_K_M49.9 GB5.1 tok/snot reported
Q6_K57.9 GB4.4 tok/snot reported
Q8_075.0 GB3.4 tok/snot reported
FP16~141 GBdoes not fit in 128 GBn/a

Strix Halo can hold quantizations of a 70B model that a 24 GB RTX 4090 cannot load without offload. For models that fit on a single 4090 or 5090, the discrete card's much wider memory bus wins on tok/s.

Prefill vs generation throughput on unified memory

Prefill (processing the input prompt before generating any tokens) is compute-bound. For Llama 3.1 70B at Q4_K_M, strix-halo-llm-perf measured 81.6 tok/s at pp512, so a 4,096-token prompt takes roughly 50 seconds before the first output token. Longer prompts are slower per token as attention cost grows. MoE models prefill far faster: the same repo measured 531 tok/s for Qwen3-Coder-Next and 156 tok/s for MiniMax M2.5.

Generation (producing each output token) is memory-bandwidth-bound. At 256 GB/s, the theoretical ceiling for the 42.5 GB Llama 3.1 70B Q4_K_M file is 256 / 42.5 = 6.0 tok/s, before KV-cache reads. The measured 5.1 tok/s sits just under it.

For comparison, the Mac Studio's M5 Max at 614 GB/s has a ceiling of 614 / 42.5 = 14.4 tok/s on the same file, and the M5 Ultra at 1.2 TB/s roughly doubles that. The bandwidth-to-capacity tradeoff is the central tension of the unified memory llm conversation.

How does multi-node clustering scale (2x, 4x)?

Clustering scales capacity well and decode speed poorly. The published measurements:

  • Model fits on one node: clustering is slower. Qwen3-Coder-Next at Q6_K_XL decoded at about 38.7 tok/s on one host and about 26.3 tok/s over two-host RPC, per strix-halo-llm-perf. gpt-oss-120b fell from 33.12 tok/s on one node to 24 tok/s across four, per Jeff Geerling. A May 2026 three-node USB4 run measured Qwen3-Coder-Next UD-Q8_K_XL (86.3 GB) at 40.1 tok/s on one node, 34.5 on two and 29.4 on three under Vulkan/RADV, per strix-halo-guide issue #12. On one box, a newer llama.cpp build (b10440, ROCm 7.x) decoded gpt-oss-120b at 49.0 tok/s in August 2026, per See Hiong.
  • Model needs two nodes: it runs, at usable speed. Qwen3.5-397B-A17B at UD-Q4_K_XL (245.3 GB) decoded at 11.76 tok/s with 147.55 tok/s pp512 prefill across two USB4-linked nodes (strix-halo-llm-perf). The repo also notes that prompt shape could crash long-context runs on that model. MiniMax-M2.7 at UD-Q4_K_XL (140.8 GB) failed to load on one node and decoded at 21.4 tok/s on two ROCm 7.2 nodes and 19.7 tok/s on three; Vulkan/RADV failed to load it at one and two nodes with a roughly 792 MiB single-allocation error, per strix-halo-guide issue #12.
  • Model needs four nodes: it runs, slowly at long context. Kimi K2.5 at UD-Q2_K_XL (375 GB) decoded at 9.45 tok/s with Flash Attention and 8.30 tok/s at an 8,192-token sequence, with 100.77 tok/s prefill at batch 4096 (AMD). The same guide hit out-of-memory errors at 8,192- and 16,384-token prompts until Flash Attention was enabled.

The practical rule: buy a second node only when the model you want will not fit in one node's 128 GB. Buy the smallest node count that fits; the same May 2026 matrix found that "even when sharding is forced, 2 nodes beats 3 nodes" (strix-halo-guide issue #12). For production serving with many concurrent users, discrete datacenter GPUs remain the answer; Strix Halo clusters fit hobbyists and small teams who need to load very large open models on a bounded budget.

Perf-per-dollar math

  • One 128 GB Strix Halo box: $3,299.00 (ACEMAGIC M1A PRO+) to $3,499.99 (GMKtec EVO-X2) in SpecPicks' tracked Amazon listings on October 12–13, 2026. It runs 230B-class MoE models such as MiniMax M2.5 at 32.8 tok/s (strix-halo-llm-perf).
  • Two boxes (~$6,600–$7,000 at those prices): enough memory for Qwen3.5-397B-A17B at 11.76 tok/s (strix-halo-llm-perf).
  • Four boxes (~$13,200–$14,000 at those prices): enough for 1T-parameter Kimi K2.5 at 9.45 tok/s (AMD). In August 2025 Geerling priced his four-node Framework cluster at "around $8,004" before rack, switch and cabling (Jeff Geerling), so memory-heavy systems now cost far more than they did a year ago.
  • Mac comparison: in the same write-up, Geerling quoted an M3 Ultra Mac Studio with 512 GB at "just under $10,000" and 16 tok/s in his giant-model comparison, against 0.7 tok/s for Llama 3.1 405B on his Framework cluster (Jeff Geerling). Apple's current Mac Studio line offers the M5 Ultra with up to 512 GB at 1.2 TB/s.

Strix Halo's case is per-capacity, not per-token: a box that loads a 230B MoE model at a few thousand dollars. A Mac Studio with more memory bandwidth and one large memory pool avoids the cluster interconnect entirely.

When to choose Strix Halo over discrete GPU + when not to

Choose Strix Halo if: your primary workload is large MoE models (100B+ total parameters) that do not fit on a 24 to 32 GB GPU; you want a single-box or small-cluster solution; you value low-power continuous operation (the RTX 4090 alone is rated at 450 W total graphics power, per NVIDIA); you want to hold the model in memory at higher quantization than discrete GPUs allow.

Choose discrete GPU if: your models fit in 24 to 32 GB at usable quantizations; you need maximum tok/s for single-user latency; you do training or fine-tuning (Strix Halo has limited training story); you have an existing CUDA software stack you do not want to port.

Choose a Mac Studio (M5 Max or M5 Ultra) if: you want unified memory with no Linux setup; you value the macOS dev experience; you want more memory bandwidth in one box than any Strix Halo cluster offers.

Bottom line

As of October 2026, Strix Halo is one of the least expensive ways to load a 230B-class MoE model in one box, and MoE models are where it performs: 32.8 tok/s for MiniMax M2.5 against 5.1 tok/s for dense Llama 3.1 70B (strix-halo-llm-perf). Clustering over llama.cpp RPC extends that to 397B and 1T-parameter models at roughly 9 to 12 tok/s (strix-halo-llm-perf, AMD), but it does not speed up a model that already fits on one node. The platform is not a 4090 replacement. It answers a different question: how to run the largest open models without a datacenter GPU.

Citations and sources

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.

Products mentioned

Products mentioned in this article

Amazon & eBay listings, plus full specs and alternatives on each product page.

As an Amazon Associate, SpecPicks earns from qualifying purchases; we also earn on qualifying eBay purchases via the eBay Partner Network. Prices shown were last tracked at crawl time and may vary — check the listing for the current price.

Frequently asked questions

What makes Strix Halo suitable for large LLMs?
The Ryzen AI Max+ 395 supports up to 128 GB of 256-bit LPDDR5x-8000 (256 GB/s) shared by the CPU and the 40-CU Radeon 8060S GPU, per AMD's spec page, so models far larger than any consumer GPU's VRAM load without CPU offload. It performs best on sparse mixture-of-experts models: per the strix-halo-llm-perf benchmark repo (February 2026), one box decodes MiniMax M2.5 at Q3_K_M at 32.8 tok/s.
How does Strix Halo compare to discrete GPUs like the RTX 4090?
The RTX 4090 has 24 GB of GDDR6X on a 384-bit bus (NVIDIA), so it is faster for any model that fits in 24 GB. A Llama 3.1 70B Q4_K_M file is 42.5 GB on Hugging Face and does not fit, while Strix Halo holds it in memory and decodes it at 5.1 tok/s per strix-halo-llm-perf, close to its 256 GB/s bandwidth ceiling of about 6 tok/s.
What are the limitations of multi-node clustering with Strix Halo?
llama.cpp RPC splits layers across machines, which adds memory but not decode speed. For models that fit on one node, clustering is slower: Qwen3-Coder-Next Q6_K_XL fell from about 38.7 to 26.3 tok/s over two-host RPC (strix-halo-llm-perf), and gpt-oss-120b fell from 33.12 to 24 tok/s across four nodes (Jeff Geerling). Real node links measured about 5 to 10 Gbps, well below USB4's 40 Gbps headline.
What workloads are best suited for Strix Halo setups?
Large mixture-of-experts models that do not fit on a 24 to 32 GB GPU. Single boxes run models such as MiniMax M2.5 (32.8 tok/s) and Qwen3-Coder-Next (42.7 tok/s) per strix-halo-llm-perf. Clusters make sense only when one box's 128 GB is too small: two nodes ran Qwen3.5-397B-A17B at 11.76 tok/s, and AMD's four-node guide ran Kimi K2.5 at 9.45 tok/s. Dense 70B models are slow at about 5 tok/s.
How does Strix Halo's unified memory benefit LLM inference?
The CPU and integrated GPU share one memory pool of up to 128 GB, so the whole model and its KV cache sit in GPU-addressable memory instead of spilling to system RAM over PCIe, as happens on a discrete card with too little VRAM. Decode speed is still bounded by the 256 GB/s bandwidth AMD lists, which is why sparse MoE models run several times faster than dense ones of similar file size.

— Mike Perry · Updated 2026-11-17

Parts this article names

Amazon Associate — prices tracked 2026-11-20, may vary.