Short answer: As of October 2026, 128GB of unified memory on the Ryzen AI Max+ 395 (Strix Halo) buys you big sparse mixture-of-experts models, not fast dense 70B ones. Per the Strix Halo toolboxes llama.cpp benchmark grid (measured May 2026), gpt-oss-120b generates at 56.6 tok/s and Qwen3.5-122B-A10B at 22.3 tok/s. A community Strix Halo guide finalized September 27, 2026 runs the ~100 GiB Qwen3-235B-A22B IQ3_M at 17.34 tok/s on Vulkan RADV versus 15.65 tok/s on ROCm 10.0, with only a 512 MiB BIOS UMA reservation, but dense Llama 3.3 70B decodes at only 4.8 tok/s. Jeff Geerling's cross-platform ai-benchmarks table shows the same dense-model limit in one test: Llama 3.1 70B runs at 4.97 tok/s on the Framework Desktop mainboard (Ryzen AI Max+ 395), against 4.71 tok/s on Dell's Pro Max with GB10 (Nvidia Spark) and 14.08 tok/s on a Mac Studio M3 Ultra. The memory bus caps all of this: 256-bit LPDDR5x-8000, or 256 GB/s theoretical (AMD spec page). The cheap-box story is also over. AMD's own Ryzen AI Halo box costs $3,999, and 128GB partner boxes list at $3,449–$4,349 after the 2026 memory-price spike. With the $3,449 Framework Desktop out of stock, the cheapest in-stock 128GB box as of October 10, 2026 is the GMKtec EVO-X2 at $3,499.99 with a 1TB SSD.
AMD Ryzen AI Max+ 395 Box (Strix Halo) for Local LLMs: What 128GB Unified Memory Actually Buys You
128GB of unified memory buys big MoE models (gpt-oss-120b at ~56 tok/s), not fast dense 70B, and AMD's Strix Halo box now costs $3,999.
As of Sept 2026, 128GB Strix Halo runs gpt-oss-120b at ~56 tok/s and Qwen3.5-122B at ~22 tok/s, but dense 70B stays near 5 tok/s at 256 GB/s.
Where this fits: the 128 GB unified tier of the local-LLM GPU picks by VRAM (one card per tier, with cited tok/s). Tiers: 8 GB · 12 GB · 16 GB · 24 GB · 48 GB+ · 128 GB unified.

Hardware at a Glance
Median generation throughput at 7–9B models (Llama 3.1 8B, Qwen 3 8B), Q4 quantization, from community-reported runs SpecPicks tracks. Each row pools runs from different sources, runtimes and models in that class, so the rows are not a matched head-to-head; where the article compares cards on the same rig, its own figures are the like-for-like result. Street price is the second-lowest listing priced within the last 24 hours at 0.5–2× launch MSRP, so no single listing sets it, and a street figure well above MSRP says so in its cell; where too few listings pass that check the cell shows a dash and the launch MSRP, which is not comparable with the street figures. Rows marked for comparison are not the cards this article is about — they are included so the throughput column has something to be read against.
| GPU | VRAM | Llama-3-8B class, Q4 | Street price | Sources |
|---|---|---|---|---|
| NVIDIA GeForce RTX 5090 | 32 GB | 224.8 tok/s4 runs · 3 sources | —too few current listings for a street figure · launch MSRP was $1,999 | SpecPicks median of 4 runs; sources: llama.cpp GitHub, DatabaseMart, Hardware Corner |
| Radeon RX 7900 XTXfor comparison | 24 GB | 138.6 tok/s9 runs · 3 sources | $1,995street, all listings · 2.0× the $999 launch MSRP | SpecPicks median of 9 runs; sources: GitHub / 1337hero, llama.cpp GitHub issue #20934, Markaicode / LocalScore |
| NVIDIA RTX A5000 24GBfor comparison | 24 GB | 133 tok/s4 runs · 3 sources | —too few current listings for a street figure · launch MSRP was $1,999 | SpecPicks median of 4 runs; sources: llama.cpp GitHub (CUDA scoreboard), knightli.com (llama.cpp CUDA…, MyAIHardware |
What changed since September 2026: the ddifofeu guide's final Bosgame M5 runs put Qwen3-235B-A22B IQ3_M (99.95 GiB) at 17.34 tok/s on Vulkan RADV vs 15.65 tok/s on ROCm 10.0 (llama.cpp build 11206) and show ~100 GiB GGUFs loading at a 512 MiB BIOS UMA split, and the 192GB Ryzen AI Max+ PRO 495 boxes now list at about $6,499–$7,399 as of October 2, 2026 for about 273 GB/s of bandwidth.
The full cross-card comparison: Best GPUs for Running Local LLMs in 2026 has the VRAM-tier table from 12 GB through 48 GB, the largest model each card fits at Q4, and measured tok/s with a source link on every row.
What changed — October 2026
- A fixed 96 GiB UMA split is no longer required for the 100 GiB class. The ddifofeu guide's September 27, 2026 conclusion is that a 512 MiB BIOS UMA reservation, paired with Linux 6.18.7, the upstream
amdgpudriver and a 126,976 MiB GTT, ran Qwen3-235B IQ2_M, IQ3_XS and IQ3_M with zero swap at 19.98, 17.78 and 17.05 tok/s. The guide notes that it changed kernel, driver, GTT and TTM together, so the exact fix is not isolated. - Vulkan still beats ROCm 10.0 on decode for a 100 GiB MoE. On the same llama.cpp commit (build 11206), Qwen3-235B IQ3_M measured 17.34 tok/s on Vulkan RADV (Mesa 26.2.3) against 15.65 tok/s on ROCm 10.0, with prefill at 198.53 vs 188.03 tok/s. The guide cautions that this is one model on one machine.
- The 192GB successor has shipped, at about twice the price. Per ComputingForGeeks (updated October 2, 2026), the Ryzen AI Max+ PRO 495 takes up to 192GB of LPDDR5X-8533 for about 273 GB/s, a 6.6% bandwidth gain over the 395. 192GB boxes list at about $6,499 (ACEMAGIC F9A), $6,599 (GMKtec EVO-X5 Pro) and $7,399 (Minisforum MS-S1 MAX-P495). The same page says no independent 495 benchmark had been published as of that date.
- A same-test 70B comparison against Nvidia's GB10. In Jeff Geerling's ai-benchmarks table, Llama 3.1 70B runs at 4.97 tok/s on the Framework Desktop mainboard (395+) and 4.71 tok/s on Dell's Pro Max with GB10 (Nvidia Spark). For dense 70B decode the two 128GB boxes are about even. A Mac Studio M3 Ultra reaches 14.08 tok/s in the same table.
- The cheapest 128GB box in stock is now $3,499.99. Per ComputingForGeeks' comparison (updated October 10, 2026), the $3,449 Framework Desktop is out of stock. The GMKtec EVO-X2 128GB/1TB is $3,499.99, the Bosgame M5 128GB/2TB is $3,599, the Minisforum MS-S1 Max 128GB/2TB is $3,879 on Amazon, and the Beelink GTR9 Pro is still $4,349.
What's changed as of September 2026
Six things have changed since this article first ran:
- AMD's in-house box has shipped, and it costs twice what was expected. The Ryzen AI Halo Developer Platform (Ryzen AI Max+ 395, 128GB LPDDR5x-8000, 2TB SSD) went on sale through Micro Center for $3,999, with local availability expected by July 10, 2026. AMD's product page markets it for "up to 200B parameter models".
- Partner boxes got more expensive, not cheaper. Framework's 128GB Desktop launched at $1,999 and was raised to $2,459 because of the LPDDR5x shortage. By September 2026 it lists at $3,449, the GMKtec EVO-X2 128GB/2TB at $3,649.99, and the Beelink GTR9 Pro at $4,349.
- The model mix moved to sparse MoE. The models that make 128GB worth having now activate 3–22B parameters per token: gpt-oss-120b, Qwen3.5-122B-A10B, MiniMax-M2.7, Qwen3.6-35B-A3B and Qwen3-235B-A22B. Their speed on this box is set by active-parameter bytes, not total size.
- The rivals moved too. NVIDIA raised the DGX Spark from $3,999 to $4,699 in February 2026. Apple shipped the Mac Studio with M5 Max (up to 128GB, 614 GB/s) and M5 Ultra (up to 512GB, 1.2 TB/s), available from September 22, 2026. That replaces the M3 Ultra comparison this article used to make.
- Several earlier numbers were wrong. The previous version listed dense 27B models at 18–22 tok/s. That is above the bandwidth ceiling for a ~17 GB Q4 file (256 / 17 ≈ 15 tok/s). It also cited "Gemma 4 9B" and "Qwen 3.6 72B", and neither exists: Gemma 4 ships as E2B, E4B, 12B, 26B A4B and 31B. All of these are replaced below with measured, cited rows.
- A 192GB successor is coming. AMD's own page references a follow-up platform with 192GB memory support. (Update, October 2026: it shipped as the Ryzen AI Max+ PRO 495; see the October section above and ComputingForGeeks' 495 vs 395 comparison.)
Current large-model rows on a 128GB Strix Halo box (September 2026)
Each ceiling below is 256 GB/s divided by the bytes streamed per token. For MoE models that figure is the active-parameter share of the file (file size × active ÷ total parameters), which is an approximation: shared layers and embeddings shift it a little. File sizes are the summed Hugging Face GGUF shards for the exact quant.
| Model (quant) | Total / active params | GGUF size (HF) | Active-bytes ceiling @ 256 GB/s | Measured decode (tg128) | Source |
|---|---|---|---|---|---|
| gpt-oss-120b (MXFP4) | 117B / 5.1B | 63.4 GB | ~92 tok/s | 56.6 tok/s (Vulkan RADV); 55.9 tok/s on a Corsair AI Workstation 300 | kyuz0 grid, community wall-power run |
| Qwen3.6-35B-A3B (UD-Q4_K_XL) | 35B / 3B | 22.4 GB | ~134 tok/s | 60.4 tok/s (Vulkan RADV) | kyuz0 grid |
| MiniMax-M2.7 (UD-Q3_K_S) | ~229B / ~10B (M2 architecture) | 93.6 GB | ~63 tok/s | 31.1 tok/s (Vulkan RADV) | kyuz0 grid |
| Qwen3.5-122B-A10B (UD-Q5_K_XL) | 122B / 10B | 91.9 GB | ~34 tok/s | 22.3 tok/s (Vulkan RADV) | kyuz0 grid |
| Qwen3-235B-A22B (IQ3_XS) | 235B / 22B | 96.9 GB | ~28 tok/s | 18.1 tok/s (Vulkan, 96 GiB UMA) | ddifofeu guide |
Every measured figure sits under its ceiling, and MoE decode lands at roughly 45–65% of it. The same rule explains the dense results later in this article. A dense 70B model streams all ~42 GB for every token, so no quantity of free memory makes it fast on a 256 GB/s bus. A 10B-active MoE streams under a fifth of that. This is the practical answer to "what does 128GB buy you?" It lets you load 90–100 GB sparse models, the gpt-oss-120b, Qwen3.5-122B and Qwen3-235B class, that no 32GB discrete card can hold, and run them at conversational speed.
Two caveats on the big rows. First, the ~90–97 GB files need a large GPU memory allocation, and the Linux memory stack decides whether you get it. On an older kernel/driver stack the ddifofeu guide found Qwen3-235B IQ3_M (~100 GiB) failed at a 64 GiB UMA split but ran at 96 GiB. Its September 27, 2026 re-test ran the same model at 17.05 tok/s with zero swap at a 512 MiB UMA split on Linux 6.18.7 with upstream amdgpu and a 126,976 MiB GTT. Second, some MoE models fall off sharply at depth. In the kyuz0 grid, MiniMax-M2.7 drops from 31.1 tok/s to 14.7 tok/s at 32k context and 8.8 tok/s at 64k. gpt-oss-120b holds 43.0 tok/s at 32k.
AMD's Strix Halo boxes and what they mean for the 128GB local-LLM tier
For two years the Mac Studio was the default answer for anyone who wanted to hold a 70B-plus model in unified memory without multi-GPU offload pipelines. NVIDIA's answer, the DGX Spark, now lists at $4,699 for 128GB. AMD's answer is the Strix Halo APU, available in partner mini-PCs since 2025 and, since June 2026, in AMD's own Ryzen AI Halo box.
It is not a workstation in the EPYC / Threadripper sense. It is a small-form-factor PC built around one APU: 16 Zen 5 cores, a 40-CU Radeon 8060S iGPU, up to 128GB of 256-bit LPDDR5x-8000, and a configurable 45–120W TDP. Beelink, GMKtec, Framework, Corsair, Bosgame and others ship boards built on the same silicon.
What it brings is unified memory that the iGPU can address directly, without a PCIe hop. Bandwidth tops out at 256 GB/s. That is well short of the M5 Max's 614 GB/s and far short of an RTX 5090's 1,792 GB/s, but it sits alongside 128GB of capacity. For local LLMs those are the two figures that matter: bandwidth governs decode speed, and capacity governs which models you can load at all.
Software is the other half. As covered below, llama.cpp's Vulkan (RADV) backend decodes fastest on this chip, while ROCm is much faster at long-context prompt ingestion. Pick your backend by workload.
Key Takeaways
- 128GB is for sparse MoE models. Per the kyuz0 grid, gpt-oss-120b runs at 56.6 tok/s and Qwen3.5-122B-A10B at 22.3 tok/s. Qwen3-235B-A22B IQ3_XS runs at 18.1 tok/s.
- Dense models are bandwidth-capped. Llama 3.3 70B Q4_K_M runs at 4.8 tok/s and Qwen3.8-27B Q4_K_M at 12.9 tok/s without speculative drafting.
- Bandwidth is the bottleneck: 256 GB/s LPDDR5x-8000. Decode speed is roughly bandwidth divided by the bytes read per token.
- Pricing has roughly doubled. AMD's box is $3,999, partner boxes are $3,449–$4,349, and the DGX Spark is $4,699.
- Wall power is modest. One community run on a Corsair AI Workstation 300 measured 33.3 W idle, 173.6 W during gpt-oss-120b generation, and 259.4 W during long prefill.
What is the AMD Ryzen AI Max+ 395 box and what does it cost?
"AMD Ryzen AI Max+ 395 box" is shorthand for the mini-PC form factor built around the Strix Halo APU. AMD's own Ryzen AI Halo Developer Platform ships in Windows 11 Pro and Linux variants with identical hardware at the same $3,999 price. It is sold through Micro Center with in-store pickup. Partner boxes use the same Ryzen AI Max+ 395 silicon and the same 128GB LPDDR5x-8000 configuration, and differ mainly in chassis, cooling and I/O.
The earlier expectation of a ~$2,000 128GB box did not survive 2026. The 128GB tier needs the densest LPDDR5x packages, and those took the biggest price hits. Framework went from $1,999 to $2,459 in its first increase, and by September 2026 its 128GB Desktop lists at $3,449. Check live pricing before buying; stock and pricing shift week to week.
How much usable VRAM does 128GB unified memory give a local LLM?
In practice you choose a split between the iGPU and the OS. The ddifofeu guide measured two firmware splits on a 128GB Bosgame M5 running Ubuntu 24.04:
- 96 GiB UMA leaves about 30 GiB visible to Linux. On the guide's original Linux 6.17 stack this was the AI-first profile needed for ~90–100 GiB GGUFs.
- 512 MiB UMA with a large GTT is the guide's final recommendation as of September 27, 2026: with Linux 6.18.7, upstream
amdgpuandamdgpu.gttsize=126976, it loaded Qwen3-235B GGUFs up to 99.95 GiB with zero swap. - 64 GiB UMA leaves roughly twice as much host RAM and suits mixed desktop use, provided models stay below about the 90 GiB class.
With a 96 GiB allocation you can load:
- gpt-oss-120b MXFP4: 63.4 GB file. Comfortable, with room for long context.
- Qwen3.5-122B-A10B UD-Q5_K_XL: 91.9 GB file. Tight but measured working.
- Qwen3-235B-A22B IQ3_XS: 96.9 GB (90.25 GiB) file. Works at 96 GiB UMA with negligible spill, per the ddifofeu guide, and at 512 MiB UMA on the corrected large-GTT stack at 17.78 tok/s.
- Llama 3.3 70B Q4_K_M: 42.5 GB file, plus a KV cache sized by context (see the context section below). Comfortable, but slow.
- Any 27B–35B model at Q4–Q8: generous headroom.
It cannot load the trillion-parameter class in any usable quant, including Qwen3.8-2.4T-A95B. Those models need several of these boxes clustered, or a much larger machine. Treat ~100 GB of weights as the practical single-box ceiling.
What tok/s can Strix Halo hit on dense 27B–70B models?
Dense models read every weight on every token, so they sit much closer to the raw bandwidth ceiling than MoE models do. Every row below is a measured community figure. The ceiling column is 256 GB/s divided by the Hugging Face file size.
| Model (quant) | GGUF size | Ceiling @ 256 GB/s | Measured decode | Backend | Source |
|---|---|---|---|---|---|
Qwen3.8-27B (Q4_K_M, Ollama qwen3.8:27b) | 17.7 GB | ~14.5 tok/s | 12.9 tok/s (no drafting) | Ollama / Vulkan RADV | strix-halo-guide |
| Gemma 4 31B (UD-Q4_K_XL) | 18.8 GB | ~13.6 tok/s | 11.5 tok/s | llama.cpp Vulkan RADV | kyuz0 grid |
| Qwen3.6-27B (UD-Q8_K_XL) | 35.3 GB | ~7.2 tok/s | 6.5 tok/s | llama.cpp ROCm 7.2.3 | kyuz0 grid |
| Llama 3.3 70B (Q4_K_M) | 42.5 GB | ~6.0 tok/s | 4.8 tok/s | llama.cpp Vulkan | ddifofeu guide |
Two observations. First, the backend matters less for decode than for prefill. For Qwen3.6-27B Q8, ROCm and Vulkan RADV land within 0.2 tok/s of each other (6.5 vs 6.3) in the kyuz0 grid. Second, 70B-class dense throughput is chat-grade at best. About 5 tok/s works for reading along with an answer, but it is far too slow for agentic loops that chain dozens of calls.
Speculative decoding (MTP or draft-model) can lift the dense 27B numbers substantially in some community stacks. Those results depend heavily on prompt type and acceptance rate, so the table above reports only plain decode.
Prefill (prompt processing) is compute-bound and faster than generation, but on dense 70B it is not fast. The ddifofeu guide measures 106 tok/s PP512 on Llama 3.3 70B Q4_K_M, so an 8k-token retrieved context takes over a minute to reach the first token. gpt-oss-120b, by contrast, prefills at 720 tok/s PP512. That is another reason MoE is the better fit for this box.
The next question: 128 GB of unified memory settles what fits; it does not settle what runs fast, and the rig most buyers weigh this box against is a pair of used RTX 3090s. AMD Ryzen AI Max 395 Box: Can a 128GB Unified-Memory APU Replace a Dual-3090 Local LLM Rig? sets the two side by side on memory bandwidth, generation speed on 27B and 70B models, and the workloads where the discrete pair still wins.
Ryzen 395 box vs DGX Spark vs Mac Studio M5 Max vs RTX 5090: which 128GB-class rig?
Measured tok/s for the rival machines on these exact quants is not yet consistently published, so the model rows below are bandwidth ceilings (bandwidth ÷ bytes per token), not measurements. Real decode typically lands at roughly 45–65% of the ceiling for MoE models and 80–90% for dense models, as the Strix Halo rows above show.
| Spec / Box | Ryzen 395 box | NVIDIA DGX Spark | Mac Studio M5 Max | RTX 5090 desktop |
|---|---|---|---|---|
| Max memory | 128 GB unified | 128 GB unified | 128 GB unified | 32 GB GDDR7 + system DRAM |
| Memory bandwidth | 256 GB/s | ~273 GB/s | 614 GB/s | 1,792 GB/s (VRAM) |
| gpt-oss-120b ceiling (2.77 GB active) | ~92 tok/s (measured 56.6) | ~99 tok/s | ~222 tok/s | Does not fit in VRAM |
| Llama 3.3 70B Q4 ceiling (42.5 GB) | ~6 tok/s (measured 4.8) | ~6.4 tok/s | ~14 tok/s | Does not fit in VRAM |
| Dense 27B Q4 ceiling (17.7 GB) | ~14.5 tok/s (measured 12.9) | ~15 tok/s | ~35 tok/s | ~101 tok/s |
| Price | $3,999 (AMD); $3,449–$4,349 (partners) | $4,699 | From $2,499 (128GB costs more) | Varies |
| Software story | llama.cpp Vulkan / ROCm | CUDA | MLX, llama.cpp Metal | CUDA, best-supported |
| Fine-tuning friendly | No | Limited | Limited (MLX-LM) | Yes, for models that fit |
Three buyers, three answers:
- The Ryzen 395 box is still the cheapest way to 128GB. It has roughly the DGX Spark's bandwidth for several hundred dollars less, and it runs the big-MoE class at conversational speed.
- The Mac Studio M5 Max has 2.4× the bandwidth. That is the difference that matters on dense 70B and on higher-bit MoE quants, but a 128GB configuration costs well above the $2,499 base price.
- The RTX 5090 wins decisively on anything that fits in 32GB, including dense 27B at roughly 7× the ceiling. It is the only realistic option here for fine-tuning and image generation at speed. It simply cannot hold the 60–100 GB models that justify a 128GB box.
Does ROCm 7 actually work for Ollama / llama.cpp on Strix Halo?
Yes. The more useful finding is that the right backend depends on the workload. The kyuz0 grid tests ROCm 6.4.4, ROCm 7.2.3, ROCm 7 nightlies, Vulkan AMDVLK and Vulkan RADV on the same Fedora 43 box (data generated May 2026):
llama.cpp, decode: Vulkan RADV is usually fastest. On gpt-oss-120b it measures 56.6 tok/s against 51.8 tok/s on ROCm 7.2.3 (about 9% faster). On Qwen3.5-122B-A10B it measures 22.3 vs 20.1 tok/s.
llama.cpp, long-context prefill: ROCm wins by a wide margin. At 32k depth, gpt-oss-120b prefills at 653 tok/s on ROCm 7 nightlies against 308 tok/s on RADV. At 64k depth the figures are 413 vs 146 tok/s. For RAG or document summarization with large prompts, ROCm can halve time-to-first-token.
Ollama: Works, and wraps llama.cpp. The strix-halo-guide documents a Vulkan/RADV Ollama route (OLLAMA_VULKAN=1) for Qwen3.8-27B that passed image, tool-call and long-retrieval checks.
vLLM / SGLang: Usable on ROCm but less mature on the gfx1151 iGPU than llama.cpp. For single-stream chat, llama.cpp remains the simpler, better-documented path. Multi-tenant serving is where the vLLM setup cost pays off.
ComfyUI / image generation: Better than it was. AMD now markets PyTorch ComfyUI workloads, including SDXL, Flux and Qwen Image, on its Ryzen AI Halo box. For heavy image-generation throughput, a discrete NVIDIA card is still the stronger choice.
Fine-tuning: Not this box's job. Use a cloud GPU or a discrete-GPU desktop.
The pragmatic recipe: install a current Linux distribution (Fedora 43 and Ubuntu 24.04 both appear in the community benchmarks), either set a 96 GiB UMA split or, on a current kernel with upstream amdgpu, keep a small UMA reservation and raise the GTT size if you plan to run 90 GB-class models, build or pull llama.cpp with both Vulkan and ROCm, and switch between them by prompt length.
What are the thermal and power-draw realities of sustained inference?
The Strix Halo APU has a configurable TDP of 45–120W (55W default), and local-LLM workloads pin the iGPU for as long as a request runs. The best public whole-system figures come from a community wall-meter run on a Corsair AI Workstation 300 running gpt-oss-120b:
- Idle: 33.3 W at the wall.
- Sustained generation (gpt-oss-120b, 55.9 tok/s): 173.6 W, or about 3.1 J per generated token.
- Sustained long prefill (16k-token prompts): 259.4 W time-weighted, with brief peaks to 267.3 W.
AMD's own electricity model for the Halo box assumes a 150W sustained "nightmare case" draw, which is consistent with those numbers. None of these boxes need a dedicated circuit; a standard 15A residential outlet is fine.
Cooling and fan noise vary by chassis because the silicon is identical. Prefer boxes with a published sustained-load review, and expect audible fans during long prefill.
Quantization matrix: tok/s and memory headroom
All rows are measured llama.cpp tg128 figures. Sizes are Hugging Face file sizes for the exact quant, and "fits" assumes a 96 GiB (≈103 GB) GPU allocation.
| Model (quant) | Type | GGUF size | Decode tok/s | Fits at 96 GiB? | Source |
|---|---|---|---|---|---|
| gpt-oss-20b (MXFP4) | MoE, 3.6B active | 12.1 GB | 79.8 | Generous | kyuz0 grid |
| Gemma 4 26B-A4B (UD-Q4_K_XL) | MoE | 17.0 GB | 54.7 | Generous | kyuz0 grid |
| Qwen3.6-35B-A3B (UD-Q4_K_XL) | MoE, 3B active | 22.4 GB | 60.4 | Generous | kyuz0 grid |
| Qwen3.8-27B (Q4_K_M) | Dense | 17.7 GB | 12.9 | Generous | strix-halo-guide |
| Gemma 4 31B (UD-Q4_K_XL) | Dense | 18.8 GB | 11.5 | Generous | kyuz0 grid |
| Qwen3.6-27B (UD-Q8_K_XL) | Dense | 35.3 GB | 6.5 | Comfortable | kyuz0 grid |
| Llama 3.3 70B (Q4_K_M) | Dense | 42.5 GB | 4.8 | Comfortable | ddifofeu guide |
| gpt-oss-120b (MXFP4) | MoE, 5.1B active | 63.4 GB | 56.6 | Comfortable | kyuz0 grid |
| Qwen3.5-122B-A10B (UD-Q5_K_XL) | MoE, 10B active | 91.9 GB | 22.3 | Tight | kyuz0 grid |
| MiniMax-M2.7 (UD-Q3_K_S) | MoE, ~10B active | 93.6 GB | 31.1 | Tight | kyuz0 grid |
| Qwen3-235B-A22B (IQ3_XS) | MoE, 22B active | 96.9 GB | 18.1 | Tight (measured working) | ddifofeu guide |
The pattern is the article's thesis in one table. A 63 GB MoE model decodes about 12× faster than a 42 GB dense model on the same box.
Spec-delta: Ryzen 395 box vs DGX Spark vs Mac Studio M5 Max vs RTX 5090 desktop
| Field | Ryzen 395 box | DGX Spark | Mac Studio M5 Max | RTX 5090 desktop |
|---|---|---|---|---|
| Memory | 128GB unified | 128GB unified | Up to 128GB unified | 32GB GDDR7 |
| Memory bandwidth | 256 GB/s | ~273 GB/s | 614 GB/s | 1,792 GB/s |
| Compute | 40-CU RDNA 3.5 iGPU + 16 Zen 5 cores | GB10 Grace Blackwell | M5 Max, up to 40-core GPU | RTX 5090 |
| APU/GPU power | 45–120W cTDP | n/a | n/a | GPU alone far above the APU class |
| Price (Sept 2026) | $3,449–$4,349 | $4,699 | From $2,499 | Varies |
| Form factor | Mini-PC | Mini-PC | Compact desktop | Mid-tower desktop |
Sources for each value are linked in the comparison table above.
Prefill vs generation: why APUs are decode-bound
Strix Halo is memory-bandwidth-bound, not compute-bound, for single-stream generation. Each decode step streams the active weights through the memory bus once per token. For dense Llama 3.3 70B Q4_K_M that is all 42.5 GB, so the ceiling is 256 / 42.5 ≈ 6 tok/s, and the measured figure is 4.8 tok/s. For gpt-oss-120b it is only about 2.8 GB of active expert weights, which puts the ceiling near 92 tok/s against a measured 56.6 tok/s.
Prefill processes many tokens per pass and reuses weights, so it leans on compute. That is where backend choice matters most: ROCm's long-context prefill is roughly twice Vulkan's on gpt-oss-120b at 32k, per the grid.
The implication is that Strix Halo is a good fit for chat and "answer this question" workflows on MoE models, and a workable fit for agentic loops on fast MoE models such as Qwen3.6-35B-A3B and gpt-oss-120b. It is a poor fit for agentic loops on dense 70B, and for batched multi-user serving.
Context-length impact at 8k / 32k / 128k
KV-cache scales linearly with context length. For Llama 3.3 70B, whose config specifies 80 layers, 8 KV heads and a head dimension of 128, an fp16 KV cache costs about 0.33 MB per token:
- 8k context: ~2.7 GB KV-cache. Weights plus KV total ~45 GB. Fits easily.
- 32k context: ~10.7 GB. Total ~53 GB. Fits easily.
- 128k context: ~43 GB. Total ~86 GB. Fits at a 96 GiB allocation, though prefill at that length takes a very long time.
This corrects the previous version, which overstated Llama 70B's KV cache by roughly 3× and said 128k context did not fit.
Decode speed also degrades with depth. Measured in the kyuz0 grid (Vulkan RADV unless noted):
- gpt-oss-120b: 56.6 tok/s at the start, 43.0 tok/s at 32k, 35.0 tok/s at 64k.
- Qwen3.5-122B-A10B: 22.3, then 20.4, then 19.0 tok/s. Very flat.
- Qwen3.6-35B-A3B: 60.4, then 49.2, then 43.2 tok/s.
- Qwen3.6-27B Q8 (ROCm): 6.5, then 6.1, then 5.8 tok/s.
- MiniMax-M2.7: 31.1, then 14.7, then 8.8 tok/s. Steep; avoid it for long-context work.
The 128GB capacity advantage shines on long-context MoE work. If your use case is "summarize this 100k-token document", gpt-oss-120b or Qwen3.5-122B on this box is genuinely competitive with anything in its price class. Just use the ROCm backend for the ingestion step.
Perf-per-dollar and perf-per-watt math
These figures use only measured Strix Halo decode rates and the AMD box's $3,999 price. Partner boxes at $3,449 improve the dollar column by about 14%.
| Model on a Ryzen AI Halo ($3,999) | Measured decode | $ per tok/s | Notes |
|---|---|---|---|
| gpt-oss-120b MXFP4 | 56.6 tok/s | ~$71 | 3.1 J/token at 173.6 W wall on a Corsair AI Workstation 300 |
| Qwen3.5-122B-A10B UD-Q5_K_XL | 22.3 tok/s | ~$179 | |
| Qwen3-235B-A22B IQ3_XS | 18.1 tok/s | ~$221 | |
| Llama 3.3 70B Q4_K_M | 4.8 tok/s | ~$831 |
On a dollars-per-token basis, dense 70B costs over 10× more than a 120B MoE on the same hardware. If your workload can use gpt-oss-120b or Qwen3.5-122B instead of a dense 70B, this box is a much better buy. If it cannot, the higher-bandwidth Mac Studio M5 Max is the more sensible 128GB machine.
Verdict matrix
- Get the Ryzen 395 box if you want the cheapest 128GB machine for large sparse MoE models (gpt-oss-120b, Qwen3.5-122B-A10B, Qwen3-235B-A22B at IQ3), are comfortable choosing between llama.cpp's Vulkan and ROCm backends, and prefer Linux.
- Get a Mac Studio M5 Max if you need faster dense-model decode (2.4× the bandwidth), value polished software (MLX, llama.cpp Metal, LM Studio), and can pay for the 128GB configuration.
- Get a DGX Spark if you need CUDA compatibility in a 128GB mini-PC and accept a $4,699 price for bandwidth similar to Strix Halo.
- Get an RTX 5090 desktop if your models stay at 32B or below, you also need image generation or fine-tuning, or you need fast dense-model decode for agentic workflows.
Bottom line
As of September 2026, the AMD Ryzen AI Max+ 395 box is the cheapest way to get 128GB of GPU-addressable memory, but it is no longer a $2,000 machine. AMD's own box is $3,999 and partner boxes start around $3,449. What 128GB buys is the large-MoE tier:
- gpt-oss-120b at about 56 tok/s
- Qwen3.5-122B-A10B at about 22 tok/s
- Qwen3-235B-A22B at about 18 tok/s
Those models cannot run on any 32GB card. What it does not buy is fast dense 70B inference: Llama 3.3 70B stays near 5 tok/s because 256 GB/s is 256 GB/s. Buy it for MoE models, use Vulkan for chat and ROCm for long prompts, and look at the Mac Studio M5 Max if dense-model speed is what you need.
Related guides
- Best GPU for AI workstation in 2026
- ROCm 2026 state-of-play for AMD local LLMs
- Best 24GB GPU for local LLM in 2026
Citations and sources
- AMD, Ryzen AI Max+ 395 specifications (cores, 40-CU Radeon 8060S, 256-bit LPDDR5x-8000, 128 GB max, 45–120W cTDP): https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html
- AMD, Ryzen AI Halo Developer Platform product page ("up to 200B parameter models", $3,999 retail, 150W sustained cost-model assumption, 192GB follow-up): https://www.amd.com/en/products/processors/desktops/ryzen/ryzen-ai-halo.html
- TweakTown, AMD Ryzen AI Halo launches at $3,999 via Micro Center (June 2026): https://www.tweaktown.com/news/112183/amds-ryzen-ai-halo-ai-mini-pc-launches-in-the-us-with-128gb-memory-and-a-dollars3999-price-tag/index.html
- ComputingForGeeks, Ryzen AI Max+ 395 mini-PC prices (September 2026): https://computingforgeeks.com/ryzen-ai-max-395-mini-pc-comparison/
- Mezha, Framework Desktop 128GB price increase from $1,999 to $2,459: https://mezha.ua/en/news/framework-desktop-prices-impacted-by-ram-shortage-307704/
- kyuz0, AMD Strix Halo toolboxes llama.cpp backend benchmark grid (data generated May 2026): https://kyuz0.github.io/amd-strix-halo-toolboxes/
- kyuz0, raw benchmark data behind the grid: https://kyuz0.github.io/amd-strix-halo-toolboxes/results.json
- ddifofeu, Strix Halo 128GB LLM guide (Qwen3-235B-A22B, Llama 3.3 70B, UMA split measurements): https://github.com/ddifofeu/strix-halo-128gb-llm-guide
- strix-halo-guide, Qwen3.8 27B on Strix Halo (12.89 tok/s without drafting, September 26, 2026): https://github.com/hogeheer499-commits/strix-halo-guide/blob/main/QWEN38_STRIX_HALO.md
- strix-halo-guide issue #6, community wall-power measurement of gpt-oss-120b on a Corsair AI Workstation 300: https://github.com/hogeheer499-commits/strix-halo-guide/issues/6#issuecomment-4414323665
- Apple Newsroom, Mac Studio with M5 Max and M5 Ultra (August 2026): https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/
- IntuitionLabs, NVIDIA DGX Spark review ($4,699 price, ~273 GB/s): https://intuitionlabs.ai/articles/nvidia-dgx-spark-review
- Hugging Face, openai/gpt-oss-120b model card (117B total, 5.1B active): https://huggingface.co/openai/gpt-oss-120b
- Hugging Face, Qwen/Qwen3.6-35B-A3B model card (35B total, 3B active): https://huggingface.co/Qwen/Qwen3.6-35B-A3B
- Hugging Face, Qwen/Qwen3.5-122B-A10B model card (122B total, 10B active): https://huggingface.co/Qwen/Qwen3.5-122B-A10B
- Hugging Face, MiniMaxAI/MiniMax-M2 model card (230B total, 10B active architecture): https://huggingface.co/MiniMaxAI/MiniMax-M2
- Hugging Face, bartowski Qwen3-235B-A22B-Instruct-2507 GGUF (IQ3_XS file sizes): https://huggingface.co/bartowski/Qwen_Qwen3-235B-A22B-Instruct-2507-GGUF
- Hugging Face, google/gemma-4-31B-it model card (Gemma 4 size lineup): https://huggingface.co/google/gemma-4-31B-it
- Hugging Face, Llama 3.3 70B Instruct config (layers, KV heads, head dimension): https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/blob/main/config.json
- geerlingguy/ai-benchmarks — LLM benchmark results (GitHub) — Llama 3.1 70B eval rate: Framework Desktop Mainboard (395+) 4.97 tok/s, Dell Pro Max with GB10 (Nvidia Spark) 4.71 tok/s, Mac Studio M3 Ultra 512GB 14.08 tok/s.
- ddifofeu, corrected-capacity data (512 MiB UMA, Linux 6.18.7: Qwen3-235B IQ2_M 19.98, IQ3_XS 17.78, IQ3_M 17.05 tok/s, zero swap; September 2026): https://github.com/ddifofeu/strix-halo-128gb-llm-guide/blob/main/data/corrected-capacity.csv
- ddifofeu, Vulkan vs ROCm 10.0 backend comparison (Qwen3-235B IQ3_M, llama.cpp build 11206: 17.34 vs 15.65 tok/s; September 2026): https://github.com/ddifofeu/strix-halo-128gb-llm-guide/blob/main/data/backend-comparison.csv
- ComputingForGeeks, Ryzen AI Max+ 495 vs Max+ 395 (192GB LPDDR5X-8533, ~273 GB/s, 495 box prices as of October 2, 2026): https://computingforgeeks.com/ryzen-ai-max-495-vs-395/
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
Products mentioned in this article
Amazon & eBay listings, plus full specs and alternatives on each product page.
As an Amazon Associate, SpecPicks earns from qualifying purchases; we also earn on qualifying eBay purchases via the eBay Partner Network. Prices shown were last tracked at crawl time and may vary — check the listing for the current price.
Frequently asked questions
What does 128GB of unified memory actually let a Ryzen AI Max+ 395 box run?
How fast does a dense 70B model run on Strix Halo?
How much does a 128GB Ryzen AI Max+ 395 box cost in September 2026?
Should I use Vulkan or ROCm for llama.cpp on Strix Halo?
How much power does a Strix Halo box draw during LLM inference?
Related reading
- Best GPU for AI Workstations in 2026
- Best 24GB GPU for Local LLM Inference in 2026
- AMD Ryzen AI Max+ 395: The 128GB Mini PC Running 70B LLMs Locally
- Strix Halo Clustering for Local LLMs: What the LocalLLaMA Reports Show
- Half an MI350X in a PCIe Slot: Inside AMD’s 144GB MI350P, the First Air-Cooled CDNA 4 Card
- AMD Ryzen AI Max 395 Box: Can a 128GB Unified-Memory APU Replace a Dual-3090 Local LLM Rig?
- AMD Ryzen AI Max+ PRO 495 192GB: What the PassMark Leak Tells Us
- AMD Ryzen AI Halo: A DGX Spark Rival at Mini-PC Size