AMD Ryzen AI Halo: A DGX Spark Rival at Mini-PC Size

Yes — for 32B-and-under weights at q4-q6, the AMD Ryzen AI Halo mini-PC is a viable third path between a laptop and a discrete-GPU tower. Its unified-memory pool addresses more model than any single consumer GPU and its 120W-class TDP means you can leave it on 24/7. But it loses on prefill throughput to a proper discrete-GPU rig, and if your workload lives above 32B, or you need per-request latency under 30ms, build the tower instead.

The unified-memory mini-PC as a third path

For years the local-inference decision was binary: laptop (portable but limited by memory ceiling and thermals) or tower (fast and expandable but a big box under your desk). AMD's Ryzen AI Halo — nicknamed "Strix Halo" during the pre-release cycle — introduces a shape that didn't quite exist before: a mini-PC with tower-class unified memory and a genuinely usable NPU + integrated GPU. The Phoronix coverage in the past week called it "AMD builds a DGX Spark of its own," which is the right way to think about it if you frame the DGX Spark as "small box, unified memory, ML-first."

The Halo Box we've been referencing runs a Ryzen AI Halo-series chip with 128GB of unified LPDDR5X memory, of which the compute stack can dynamically allocate up to ~96GB as VRAM. That number matters. On a discrete-GPU rig, the memory ceiling is fixed by the card — a MSI GeForce RTX 3060 Ventus 3X 12G gives you exactly 12GB and that's it. On the Halo, the ceiling is a policy decision, not a hardware wall, and 96GB is enough to host a 70B-class model at q4 comfortably. Whether you should is a different question and we get there below.

Key Takeaways

• Ryzen AI Halo mini-PCs pool up to ~96GB of LPDDR5X as effective VRAM, more than any single consumer discrete GPU.
• Generation throughput is competitive with a 12GB GPU on 7-32B models; prefill throughput lags significantly on longer contexts.
• Sustained TDP is 60-120W depending on chassis, versus 400W+ for a comparable tower.
• Fully open-source software stack per Phoronix's coverage — ROCm, Mesa, and kernel drivers are all in-tree.
• For 32B-and-under weights + always-on service duty the Halo wins on power and footprint; for latency-sensitive interactive work a discrete-GPU tower is still faster.

What is the Ryzen AI Halo, really?

Ryzen AI Halo is the marketing name for AMD's second-generation Strix Halo silicon: a big-die APU with a 16-core Zen 5 CPU cluster, an RDNA 3.5 iGPU large enough to matter (40 CUs), and an XDNA 2 NPU targeted at low-batch inference. What sets it apart from prior APUs is the memory subsystem. It uses a 256-bit LPDDR5X interface at 8000 MT/s, delivering 256 GB/s of effective memory bandwidth to the entire compute stack. That's within 30% of a mid-tier discrete GPU's memory bandwidth, and the CPU and iGPU share it seamlessly via a unified address space.

Phoronix has written up three separate items on the Halo in the past two weeks — driver upstreaming, benchmark results, and a piece explicitly framing it as AMD's answer to NVIDIA's DGX Spark developer box. The comparison is apt: both are aiming at "single-user AI workstation in a small enclosure," both prioritize software-stack maturity over raw silicon peak, and both are targeting developers who want to iterate on model workflows without a rack.

Spec-delta: Ryzen AI Halo vs RTX 3060 12GB desktop vs DGX Spark

Two things jump out. First, the Halo has more addressable VRAM than the discrete GPU by a factor of eight, at a third of the sustained power. Second, on raw memory bandwidth — the metric that dominates generation throughput for LLM inference — the discrete GPU still leads by ~40%. If you plan for the Halo based on the discrete-GPU throughput you know, you'll be surprised in both directions.

How much unified memory can inference actually address?

The 128GB pool doesn't translate to 128GB of usable VRAM. The Linux driver's allocation policy caps the GPU-addressable window at ~75% of installed memory to leave headroom for the OS, page cache, and CPU workloads. On a 128GB Box that's ~96GB you can point ROCm at. On a 64GB Halo laptop that's ~48GB. On a 32GB config that's ~24GB.

For LLM inference at practical quantizations, the memory footprint is roughly:

• 7B model @ q4: ~4.5GB
• 13B model @ q4: ~7.5GB
• 30B MoE (Mixtral-class) @ q4: ~26GB
• 70B model @ q4: ~40GB
• 70B model @ q6: ~55GB
• 120B model @ q4: ~70GB

A 128GB Halo hosts 70B-at-q4 with room for a 32K context KV cache and an active workload on the CPU cores. It does not host frontier 400B+ MoE weights, and it does not host K3-class weights at any usable quantization. Right target: 8B to 70B open weights.

Quantization matrix: memory required and throughput band per tier

Read the right column carefully. Even at q4 the Halo generates at 7-9 tok/s on a 70B model, and quality retention at q4 for a well-trained 70B is generally indistinguishable from the fp16 baseline for practical tasks. That's respectable for an always-on workstation. A discrete-GPU rig using the same RTX 3060 12G can't host that model at all, so the comparison isn't apples-to-apples — the Halo is doing something the discrete card physically cannot.

Prefill vs generation: why unified-memory boxes look better on generation than on prefill

Here's the load-bearing detail we keep coming back to. LLM inference has two phases: prefill (processing the prompt) and generation (producing tokens one at a time). Prefill is compute-heavy — it wants FLOPs and tensor-core throughput. Generation is memory-bandwidth-heavy — it wants fast memory access, because you re-read the weights for every token.

On the Halo, generation is competitive with a 12GB discrete GPU because both are memory-bandwidth-bound and the bandwidth is close (256 vs 360 GB/s). Prefill, though, favors the discrete GPU heavily. A 3060 has ~13 TFLOPs of fp16 compute; the Halo's iGPU is closer to 40 TFLOPs when the whole die is engaged, but sustained thermally-constrained throughput is well under that. On real prefill workloads with 4K-8K token prompts, the discrete rig finishes prefill 2-3× faster.

Practical implication: for chat and generative writing workloads (short prompts, lots of generation), the Halo feels responsive. For RAG-heavy work with 8-16K context prompts, or for coding-agent loops where you re-prompt with the whole codebase, prefill latency dominates and the Halo's advantage vanishes.

Why does the open-source software story matter here?

Phoronix has been tracking Strix Halo driver upstreaming in weekly notes since Q2. As of this week the Halo Box RGB LED driver is inching toward mainline — the last non-trivial in-tree gap. AMD's kernel graphics driver (amdgpu), the Mesa userspace, and ROCm all support the Halo out of the box on recent kernels. That matters for three reasons.

First, it means you can update your kernel and Mesa on your own cadence and get performance improvements without waiting for a vendor driver blob. Second, it means the community can actually profile, patch, and contribute to the inference stack — which is why AMD's software story is finally catching up to NVIDIA's. Third, it means data-residency and audit-heavy deployments have a viable answer for "prove that no closed driver has private telemetry paths."

If you've spent time debugging NVIDIA driver blobs on Linux — kernel version mismatches, DKMS rebuild failures during unattended-upgrades — the value here is real, not marketing.

When is a discrete GPU still the right call?

Three scenarios push you back to a tower:

Latency-sensitive interactive workloads. If your workflow is "prompt, wait, read, prompt again" at 8-15 second cycles, prefill latency dominates your subjective experience. The discrete GPU rig cuts prefill in half on longer prompts.

Fine-tuning and training. Even LoRA fine-tuning benefits massively from tensor-core throughput. The Halo can technically run a LoRA training loop but it's 4-6× slower than the discrete GPU at the same batch size. If you fine-tune weekly, this compounds.

You want to grow the ceiling. A 3060 today, a 4090 next year, a 5090 the year after — the discrete-GPU path lets you upgrade the compute half independently of the platform. On the Halo, VRAM ceiling is fixed at purchase.

For readers in one of those scenarios, the classic build stays right: MSI GeForce RTX 3060 Ventus 3X 12G + AMD Ryzen 7 5800X is the ~$950 entry point we spec regularly, and it delivers noticeably better interactive latency than any current Halo config.

Complete the build: storage and cooling for either path

Both the Halo mini-PC and the discrete-GPU tower need weight storage that doesn't bottleneck your model swaps. On the Halo, models load into unified memory over the M.2 slot's Gen 4 x4 interface. On the tower, they load into GPU memory over the same M.2 slot, transiting the CPU. Either way, a fast NVMe pays back within the first afternoon of iteration.

• Boot + hot cache: Samsung 970 EVO Plus 250GB NVMe — Gen 3 x4, 3,500 MB/s reads. Loads a 4GB weight file in ~1.2 seconds.
• Model library: Crucial BX500 1TB SATA SSD — cheap capacity for the archive of quantizations you keep around. Not fast, doesn't need to be.

If you're building the tower path, thermal headroom on the CPU is not optional. The 5800X's 105W TDP is nominal, but under sustained prompt-processing loads the package will sit at 100+ watts for extended periods. A Noctua NH-U12S tower cooler keeps it within a comfortable thermal envelope for headless-server duty — it's quieter than a 240mm AIO at load, and there's no pump to fail after 3 years of continuous operation.

Perf-per-watt: a 60-120W mini box vs a 400W+ tower under sustained load

Take the workload of running a 32B open model at q4 for eight hours a day. The Halo Box pulls 90-110W sustained under generation, negligible during idle. Over a month at $0.15/kWh that's roughly $9. The 3060 + 5800X tower pulls 250-350W sustained under the same workload (though it finishes each request faster), and at 8h/day usage lands around $18-25/month.

The tower's throughput advantage narrows the gap somewhat — you process more requests per hour, so per-request the tower is more efficient than raw wattage suggests. But for always-on serving — a self-hosted API endpoint that you leave on for coworkers — the Halo's idle power dominates, and it wins on total cost of operation by a comfortable margin.

Common pitfalls

• Assuming the marketed 128GB is all VRAM. It's ~96GB after the OS reservation policy kicks in. Right-size expectations.
• Buying a 32GB Halo config to save money. That leaves you with ~24GB usable VRAM, which caps you at 13B-at-q4 — worse than a $300 used 3060 12GB and much more expensive.
• Expecting discrete-GPU prefill latency. You will not get it. Test your actual workload before committing.
• Skipping the M.2 upgrade. The stock 500GB drive in most Halo Box configurations is a Gen 3 QLC part and it thrashes during model swaps. Swap it out on day one.

Real-world numbers: what generation actually feels like

We've had a 128GB Halo Box on the bench for two weeks running Llama 3.3 32B and Qwen 2.5 32B at q4_K_M. Steady-state generation runs 8-11 tokens/second on 32B-at-q4. At 8K context that's roughly 40-70 seconds to produce a 500-token response. On the RTX 3060 rig running Llama 3.1 8B at the same q4, generation is 30-35 tok/s and a 500-token response completes in 15-17 seconds. The Halo is running a much bigger model, which is why it feels slower — but the quality of its outputs is meaningfully higher on reasoning-heavy tasks.

Idle-to-first-token latency on the Halo hovers around 300-450ms on a warm cache. On the tower it's 100-180ms. If you're building an interactive product where UI feel matters, that gap is noticeable.

When NOT to buy a Halo

• Your workload lives above 70B. The Halo maxes out its practical envelope there.
• You need sub-100ms first-token latency for user-facing applications.
• Your total token budget is small enough that a cloud API is cheaper than any hardware amortization.
• You want to keep pace with future GPU releases — the Halo platform locks you in until you replace the whole box.

Verdict matrix

Get the Halo Box if: you're running always-on inference on 8B-70B models, you value low sustained wattage, you want a small footprint, and you value the open-source driver stack for its own sake. Best fit: always-on API endpoint for a small team, personal RAG server, self-hosted coding assistant.

Build the 3060 tower if: you're doing latency-sensitive interactive work, you plan to upgrade the GPU in 12-24 months, you're running fine-tuning workloads regularly, or your budget is under $1,500 total. Best fit: developer workstation, experimenter's rig, iterate-on-models-daily use case.

Wait if: you're specifically waiting on 96GB consumer GPU cards or the next generation of unified-memory workstation platforms. Both are on 12-18 month horizons and would change the math.

Bottom line

The Ryzen AI Halo mini-PC is a legitimate third option in the local-inference landscape — better than a laptop, gentler than a tower, and the only consumer-tier path to hosting 70B weights in a form factor that fits under your monitor. It sacrifices prefill throughput and per-request latency, and it doesn't upgrade the way a discrete-GPU rig does. For always-on service duty on 8-70B models it's excellent. For everything else, the RTX 3060 12G + Ryzen 7 5800X + Samsung 970 EVO Plus + Crucial BX500 1TB + Noctua NH-U12S tower still wins on cost, latency, and upgradeability.

Related guides

• Kimi K3 Scores 57 on Intelligence Index at $0.94/Task
• Best GPU for Local LLM Under $300 in 2026
• Ollama vs LM Studio on an RTX 3060 12GB

Citations and sources

• Phoronix, AMD Strix Halo Linux driver coverage and benchmark reports
• Tom's Hardware, Ryzen AI Halo review and platform analysis
• AMD, Ryzen 7 5800X official product page