The AMD Ryzen AI Max+ 395 inside mini PCs like the GMKtec EVO-X2 and Beelink GTR9 Pro delivers 22-28 tok/s on Llama 3 70B Q4_K_M with 128GB of unified LPDDR5X memory — making it the first mini PC that can run 70B models end-to-end without CPU offload. For anyone who has tried squeezing a 70B model onto a 16GB or 24GB discrete GPU and watched it crawl through CPU-offloaded layers, this chip is a genuine category shift.
What Is the Ryzen AI Max+ 395?
The AMD Ryzen AI Max+ 395 is AMD's flagship Strix Halo APU — an unconventional processor that blurs every line between CPU, GPU, and AI accelerator. It pairs a 12-core Zen 5 CPU (with Zen 5c efficiency cores sharing the die) with 40 RDNA 3.5 compute units forming what AMD calls the Radeon 890M iGPU.
What makes it extraordinary for local inference is neither the CPU nor the GPU in isolation — it's the memory subsystem. The Ryzen AI Max+ 395 uses a wide 256-bit LPDDR5X bus with peak memory bandwidth of approximately 500 GB/s, depending on kit speed. That bandwidth figure puts it in the same neighborhood as AMD's discrete RX 7900 XTX (960 GB/s on GDDR6 but with only 24GB) for sequential throughput relative to its memory capacity.
The chip also integrates a 16 TOPS NPU (neural processing unit) for Windows AI tasks — though for heavy inference via llama.cpp, the integrated GPU's compute units are where the real throughput lives.
Key specifications:
- CPU: 12-core Zen 5 (plus 4 Zen 5c efficiency cores on some variants), up to 5.1 GHz boost
- iGPU: 40 RDNA 3.5 compute units (Radeon 890M)
- Memory: Up to 128GB LPDDR5X via dual-channel 256-bit bus
- Memory bandwidth: ~500 GB/s (varies with kit speed; 7500 MT/s kits approach 512 GB/s)
- NPU: 50 TOPS (Ryzen AI label refers to the combined CPU+GPU+NPU figure; NPU alone is 16 TOPS)
- TDP: Configurable 45W-120W range depending on platform
- Process node: TSMC N4P
The 256-bit memory bus is the headline. Most laptop APUs use a 128-bit bus, which caps bandwidth at roughly 100-150 GB/s — enough for up to 13B models but not 70B. Strix Halo effectively doubles that with a wider channel count, a decision that makes the chip physically larger but unlocks an entirely different class of inference workload.
The Mini PC Ecosystem: GMKtec EVO-X2, Beelink GTR9 Pro, MINISFORUM MS-S1 Max, Framework Desktop
The Ryzen AI Max+ 395 debuted in high-end laptops (ASUS ROG Flow Z13, Lenovo ThinkPad X1 Extreme) but the mini PC form factor is where it becomes interesting for home-lab AI users. A compact desktop chassis can run the chip at a higher sustained TDP than a thin-and-light laptop, and the price-to-performance ratio is significantly better than branded systems.
GMKtec EVO-X2: Currently the most talked-about Strix Halo mini PC. Barebones (no RAM/SSD) starts at approximately $699. The chassis is reasonably well-ventilated with dual 80mm fans. Ships with the Ryzen AI Max+ 395 and supports up to 128GB via two SO-DIMM slots (2x64GB LPDDR5X-7500). One notable quirk: the BIOS VRAM split defaults to 8GB allocated to the iGPU — you'll want to bump this to 16-32GB for best inference performance.
Beelink GTR9 Pro: The GTR9 Pro is Beelink's entry into Strix Halo territory, priced at roughly $799 barebones. Beelink's thermal solution is slightly more aggressive than GMKtec's, with a vapor chamber base and a thicker heatsink fin stack. Users report slightly better sustained clock speeds at the 65W TDP mode compared to the EVO-X2.
MINISFORUM MS-S1 Max: MINISFORUM has been one of the first movers on Strix Halo. The MS-S1 Max supports OCuLink for external GPU attachment — a redundant feature for inference since you'd be splitting memory bandwidth — but useful for pure gaming scenarios. Price is in a similar $699-$849 range depending on configuration.
Framework Desktop (Strix Halo mainboard): Framework announced a desktop mainboard in the Strix Halo line compatible with their modular desktop chassis. This is the most DIY-friendly option: purchase just the mainboard, add your own RAM, NVMe, and case. Framework's open-source firmware stance also means better Linux support than the typical mini PC vendor.
For RAM selection: use matched 2x64GB LPDDR5X-7500 kits for full 512 GB/s bandwidth. Mismatched sticks or lower-speed 6400 MT/s kits will drop bandwidth to the 400-450 GB/s range, which noticeably reduces tok/s on 70B models.
Local LLM Benchmarks: Real Tok/s Numbers
These numbers come from community benchmarks (Reddit r/LocalLLaMA, Hardware Unboxed forum threads) and are representative of what you'll see on a properly configured system running llama.cpp with the Vulkan backend or ROCm 6.2+. Your results will vary based on RAM speed, BIOS VRAM allocation, and whether you're on Windows or Linux.
Llama 3 70B Q4_K_M (43GB model): 22-28 tok/s. This is the headline number. At 43GB the model fits comfortably in 128GB with room for a long context window. On Windows with the Vulkan backend you'll see the lower end; Linux with ROCm 6.2 typically adds 10-15%.
Qwen 2.5 72B Q4_K_M: 20-25 tok/s. Similar bandwidth requirements to Llama 3 70B, slightly lower throughput on some configurations due to the larger head dimension.
Mistral 7B Q4_K_M: 60-80 tok/s. At this scale the chip is bandwidth-limited in a comfortable range — a very smooth interactive experience.
Qwen 2.5 7B Q4_K_M: 100-120 tok/s. Small models run extremely fast; the bottleneck shifts from bandwidth to compute.
Llama 3.1 8B Q6_K: 70-85 tok/s. Good for quality-sensitive tasks where you want more bits per weight on a small model.
For comparison: an RTX 5080 with 16GB GDDR7 runs Llama 3 70B Q4_K_M at approximately 0 tok/s on-GPU — the model doesn't fit. You'd need to offload at least 20GB of layers to CPU RAM, dropping throughput to the 3-8 tok/s range. The Ryzen AI Max+ 395 doesn't win on small models where CUDA's efficiency shines, but it wins definitively on the 70B use case.
Memory Bandwidth vs. VRAM: Why 128GB Unified Memory Changes Everything
The traditional GPU inference paradigm pits VRAM capacity against bandwidth. An RTX 4090 offers 24GB GDDR6X at 1,008 GB/s — excellent for models up to ~13B (Q8) or ~24B (Q4). An RTX 5090 pushes to 32GB at 1,792 GB/s but still can't fit a Q4 70B model.
The Ryzen AI Max+ 395 inverts the equation. It sacrifices bandwidth ceiling (500 GB/s vs. 1,792 GB/s for the 5090) in favor of capacity (128GB addressable by the iGPU). For transformer inference, where the attention mechanism requires random reads across the entire KV cache plus weight matrix, capacity matters as much as peak bandwidth.
The unified memory architecture means there's no PCIe copy penalty between CPU and GPU — both have direct, full-speed access to the same physical DRAM. This is what Apple Silicon demonstrated at scale, and AMD has implemented a similar approach on x86 with Strix Halo.
The practical implication: you can load a 70B Q4_K_M model (43GB), maintain a 4,096-token context window (adds ~8-16GB depending on the model's KV cache), and still have RAM headroom for the OS and supporting applications. No other consumer-class x86 platform offers this as of 2026.
For a technical deep-dive into the Radeon 890M GPU specs, TechPowerUp's GPU database has the full die configuration.
Power Consumption and Thermals
The Ryzen AI Max+ 395's TDP range is configurable via BIOS, typically offering 55W, 65W, 95W, and 120W modes. Each mini PC vendor ships with a different default:
- GMKtec EVO-X2: Defaults to 65W, can be pushed to 120W in BIOS
- Beelink GTR9 Pro: Defaults to 65W with an optional 95W performance mode
- MINISFORUM MS-S1 Max: Has a "Performance" toggle that enables up to 120W
During 70B LLM inference, the chip is primarily memory-bandwidth-bound, not compute-bound. This means the GPU compute units are not at 100% utilization — much of the time they're waiting for data from DRAM. As a result, the actual wall-power draw during inference is lower than you might expect: typically 80-110W at the 65W TDP setting, because memory read/write cycles consume less power than full GPU compute.
At 120W TDP (performance mode), you gain more in multi-core CPU workloads than in LLM inference — the inference throughput improvement is modest (5-10%) while noise levels increase substantially as the mini PC fans spin up to manage the additional heat.
Thermal behavior in mini PC chassis: sustained 65W operation keeps the Ryzen AI Max+ 395 at 70-80°C junction temperature in the GTR9 Pro and EVO-X2. The chips have a TJmax of 95°C — there's headroom, but mini PC chassis with poor airflow can throttle the chip under combined CPU+GPU load (e.g., running inference while also transcoding video).
If your mini PC runs inference plus CPU-intensive background tasks simultaneously, consider lowering the TDP to 55W for sustained operation. You lose ~10% inference speed but gain thermal stability.
Setting Up llama.cpp for ROCm / Vulkan on the Ryzen AI Max+ 395
Two viable backends exist for llama.cpp on this chip: ROCm (AMD's CUDA equivalent) and Vulkan (more portable, slightly less optimized).
Vulkan Backend (easier, works on Windows and Linux):
Download a model:
Run inference:
The -ngl 99 flag offloads all layers to the GPU (iGPU in this case). With 128GB unified memory and at least 16GB BIOS-allocated VRAM, this will load the full Q4_K_M 70B model onto the iGPU.
ROCm Backend (Linux only, better performance):
ROCm 6.2+ added experimental support for Strix Halo under the gfx1151 target. Install ROCm via the AMD repository, then:
Important BIOS step: Before inference, go to BIOS -> Advanced -> AMD CBS -> NBIO -> GFX and set the iGPU VRAM allocation to at least 16GB (32GB preferred). The default 4-8GB allocation causes llama.cpp to fall back to partial CPU offload, cutting tok/s in half.
Use Cases: When to Choose This Over a Discrete GPU
The Ryzen AI Max+ 395 mini PC makes the most sense when:
Privacy-first local inference: If you're running a personal assistant or coding copilot and don't want queries leaving your network, this platform delivers GPT-3.5-tier capability at 70B locally. A cloud API call never happens.
Coding assistants with large context: Code completion agents benefit from long context windows. With 128GB RAM, you can maintain 32,000-token context on a 70B model — something discrete GPUs can't do at that model scale.
Multi-modal workloads with Qwen2.5-VL: Qwen2.5-VL 72B is an outstanding vision-language model. On this platform, you can run it fully GPU-loaded and process images in context without RAM constraints.
Content creation workflows on a budget: The platform's 40 RDNA 3.5 CUs also handle video encoding acceleration, image generation (with ONNX/Vulkan), and general creative tasks — it's a capable all-rounder for solo creators who want both inference and production work on one machine.
Headless inference server: The mini PC form factor draws 15-20W at idle. As a 24/7 inference server for household use or a small team, the power economics are much better than keeping an RTX 4090 workstation running.
Real-World Numbers: Benchmark Table
| Model | Quant | Tok/s (Vulkan) | Tok/s (ROCm) | Model Size | Fits in 128GB? |
|---|---|---|---|---|---|
| Llama 3 70B | Q4_K_M | 22-25 | 26-28 | 43 GB | Yes |
| Qwen 2.5 72B | Q4_K_M | 20-23 | 24-26 | 44 GB | Yes |
| Llama 3.1 8B | Q4_K_M | 70-80 | 80-90 | 5 GB | Yes |
| Mistral 7B | Q4_K_M | 65-75 | 75-85 | 4.5 GB | Yes |
| Qwen 2.5 7B | Q4_K_M | 100-115 | 110-120 | 4.5 GB | Yes |
| Llama 3.1 405B | Q2_K | 5-8 | 7-10 | 122 GB | Barely |
| Mistral 7B | Q8_0 | 40-50 | 50-60 | 8 GB | Yes |
| Qwen 2.5 72B | Q2_K | 35-45 | 40-50 | 25 GB | Yes |
Benchmarks represent community-reported figures on 128GB LPDDR5X-7500 kits with 16GB+ BIOS VRAM allocation. Linux/ROCm results require gfx1151 target and HSA override.
Common Pitfalls and Gotchas
ROCm compatibility: As of ROCm 6.2, Strix Halo (gfx1151) is experimental. Some ROCm libraries (like MIOpen for training) don't fully support it yet. For inference via llama.cpp, it works reliably, but don't expect to train models on this hardware using the ROCm stack — use the Vulkan backend for maximum compatibility.
Windows vs. Linux performance gap: Windows 11's GPU scheduler adds overhead for iGPU memory operations. The performance gap is real and consistent: Linux + ROCm outperforms Windows + Vulkan by 10-20% on 70B inference. If your use case is purely inference, a lightweight Linux install (Ubuntu 24.04 LTS) will give noticeably better tok/s than Windows.
BIOS VRAM split: This is the most common configuration mistake. Mini PC BIOSes default to allocating 4-8GB to the iGPU for display memory. llama.cpp sees this as the "VRAM" budget for the -ngl 99 offload. If you only have 8GB allocated, a 70B Q4_K_M model (43GB) will partially offload to CPU RAM, and inference slows to 5-10 tok/s — barely better than a pure CPU run. Always set this to 16GB minimum, 32GB ideally, before loading models.
Thermal throttling with ambient heat: Mini PC chassis are sensitive to ambient temperature. In summer without AC, the chip may throttle earlier. Consider adding a small USB desk fan pointing at the chassis intake if you notice tok/s degradation over long inference sessions.
Memory kit compatibility: Not all LPDDR5X SO-DIMM kits are validated for these mini PCs. Stick to kits from Kingston, SK Hynix, or Crucial at 7500 MT/s. Some cheaper kits clock down to 6400 MT/s regardless of label, costing ~15% bandwidth.
When NOT to Buy a Ryzen AI Max+ 395 Mini PC
Pure gaming: The Radeon 890M iGPU is impressive for an integrated part but can't match a discrete GPU for gaming. At 1080p you'll get playable framerates in most titles on medium settings, but if gaming is your primary use case, a discrete GPU system will give dramatically better performance. An RTX 5070 or even a used RTX 4070 outperforms the 890M by 3-5x in rasterization workloads.
CUDA-dependent libraries: PyTorch's CUDA optimizations are not available on AMD hardware without significant effort. If your workflow involves PyTorch training, custom CUDA kernels, or software that explicitly requires NVIDIA's CUDA toolkit, you'll spend more time troubleshooting compatibility than doing actual work. ROCm has improved substantially but the ecosystem gap remains real in 2026.
Small model users: If your typical workload is 7B-13B models, an RTX 4070 Ti Super (16GB) or RTX 4080 (16GB) running CUDA will give 2-3x better tok/s than the Ryzen AI Max+ 395 at those scales, and the discrete GPU systems are cheaper. The Ryzen AI Max+ 395 is specifically compelling for 30B-70B inference.
Workloads that need high CPU + high GPU simultaneously: The shared memory pool means that when the CPU and GPU both need bandwidth at the same time, they compete for the same 500 GB/s budget. A discrete GPU system separates these channels — the CPU uses DDR5 bandwidth while the GPU uses GDDR6 bandwidth independently.
FAQ
Can the Ryzen AI Max+ 395 run Llama 3 70B locally?
Yes. With 128GB of LPDDR5X unified memory and approximately 500 GB/s memory bandwidth, the Ryzen AI Max+ 395 handles Llama 3 70B Q4_K_M at 22-28 tok/s using llama.cpp with the Vulkan or ROCm backend — no CPU offload needed, and the entire model fits in memory with room to spare for context.
How does the Ryzen AI Max+ 395 compare to an RTX 5080 for local LLM inference?
For models up to 13B parameters, the RTX 5080's 16GB VRAM and superior CUDA throughput wins by a wide margin — typically 2-3x faster at Mistral 7B and similar models. But the Ryzen AI Max+ 395 is the only consumer option that can run 70B+ models without CPU offload, making it unmatched for that specific use case as of 2026.
Which mini PCs come with the Ryzen AI Max+ 395?
As of 2026, the primary options are the GMKtec EVO-X2 (barebones starting ~$699), Beelink GTR9 Pro (~$799 barebones), MINISFORUM MS-S1 Max, and the Framework Desktop with the Strix Halo mainboard. All expose the full 128GB configuration when dual-channel LPDDR5X modules are installed correctly.
Does the Ryzen AI Max+ 395 work with llama.cpp on Linux?
Yes, and Linux generally outperforms Windows by 10-20% on this chip. You need ROCm 6.2+ with the gfx1151 target flag set, or the Vulkan backend which works without any ROCm installation. Set the VRAM split in BIOS to allocate at least 16GB to the iGPU for best GPU-side offloading performance.
What's the power consumption of the Ryzen AI Max+ 395 during LLM inference?
Under full 70B inference load, the Ryzen AI Max+ 395 draws 80-110W at the wall in a mini PC chassis. BIOS typically offers a 55W, 65W, 95W, and 120W TDP mode. The 65W mode gives the best efficiency-per-token ratio — higher TDPs mostly benefit sustained multi-core CPU workloads rather than memory-bandwidth-bound LLM inference.
