One always-on machine, two workloads, and a single 12GB memory pool that both of them want at the same time.
Five parts for a hybrid inference and media box, sized around VRAM contention: what the encoder takes, what the model needs, and where not to spend.
Median generation throughput at 7–9B models (Llama 3.1 8B, Qwen 3 8B), Q4 quantization, from community-reported runs SpecPicks tracks. Each row pools runs from different sources, runtimes and models in that class, so the rows are not a matched head-to-head; where the article compares cards on the same rig, its own figures are the like-for-like result. Street price is the second-lowest listing priced within the last 24 hours inside a sane band of MSRP, so no single listing sets it; where too few listings pass that check the row shows launch MSRP instead. Rows marked for comparison are not covered by this article — they are the nearest cards by VRAM, included so the throughput column has something to be read against.
| GPU | VRAM | Llama-3-8B class, Q4 | Street price | Sources |
|---|---|---|---|---|
| GeForce RTX 3060 12 GB | 12 GB | 55 tok/s21 runs · 9 sources | $329MSRP | SpecPicks median of 21 runs; sources: TYO Lab blog, Ajit Singh / Hardware-Corner, Hardware Corner, llama.cpp GitHub Discussion #10879 +5 more |
| GeForce RTX 4070 SUPERfor comparison | 12 GB | 60.6 tok/s10 runs · 6 sources | $969street, all listings | SpecPicks median of 10 runs; sources: llmrun.dev, Hardware Corner, LocalScore.ai, llama.cpp GitHub Discussions +2 more |
| Arc B580for comparison | 12 GB | 41 tok/s11 runs · 9 sources | $249MSRP | SpecPicks median of 11 runs; sources: Compute Market, llama.cpp GitHub Discussions, dev.to, InsiderLLM +5 more |
See the SpecPicks review methodology.
By Mike Perry · Published 2026-09-17 · 11 min read
Most "best parts for a local LLM" guides assume the machine does one thing. The reader on a budget rarely has that luxury — there is one always-on box under the desk, and it is expected to answer prompts, serve a media library, and not sound like a hair dryer while doing it.
That consolidation creates one genuine engineering problem and a lot of false ones. The real problem is VRAM contention: model weights, the KV cache and the encoder's frame buffers all live in the same memory, and only one of those three shrinks on demand. The false problems are the ones people spend money on — a faster CPU for "transcoding headroom" on a library that mostly direct-plays, or an NVMe drive to shave seconds off a model load that happens once a day.
This guide picks five parts against that reality, with the throughput figures cited rather than asserted. The overall winner is the part that solves the contention problem instead of papering over it.
| Pick | Best For | Key Spec | Price Range | Verdict |
|---|---|---|---|---|
| MSI RTX 3060 12GB | Best overall | 12 GB GDDR6, ~360 GB/s, 170 W | Mid | The only pick that solves VRAM contention |
| AMD Ryzen 5 5600G | Best value | 6C/12T, Radeon iGPU, 65 W | Low | iGPU display output frees the whole card |
| AMD Ryzen 7 5800X | Heavy transcoding | 8C/16T, 105 W | Low-mid | CPU fallback when NVENC is saturated |
| Noctua NH-U12S | Performance add-on | 158 mm tower, no pump | Low | Nothing to fail on a box that never sleeps |
| Crucial BX500 1TB | Budget pick | SATA, up to 540 MB/s | Low | Model library storage, not a bottleneck |
12 GB GDDR6 · 192-bit · ~360 GB/s · 170 W · 7th-gen NVENC
Pros: 12 GB holds a 14B model resident with room left over; a dedicated encoder block means transcoding does not steal shader time from inference; 170 W is modest for a 24/7 box; the widest public benchmark coverage of any budget inference card.
Cons: Ampere-era efficiency is behind current generations; the 8 GB variant shares the name and is not the same product; encoder and model compete for the same memory pool.
This is the pick because it is the only one that addresses the actual conflict. Per TechPowerUp the card carries 12 GB on a 192-bit bus for roughly 360 GB/s, and NVIDIA's support matrix lists it as a 7th-generation NVENC part with one encoder chip — the encode work runs on fixed-function silicon, not the shaders your model is using.
On the inference side, Hardware Corner's RTX 3060 12GB benchmarks measure an 8B model at 55.20 tok/s using 6.0 GB at 4K context and 42.00 tok/s using 7.5 GB at 16K. llmrun.dev's database adds 12B-class models at 29.00 tok/s using 8.1 GB and 14B at 24.60 tok/s using 9.5 GB. Those VRAM figures are the important column: an 8B model at a working context leaves four to six gigabytes free, which is the headroom concurrent encode sessions need.
Buy the 12 GB part specifically — the 8 GB RTX 3060 runs a 128-bit bus and cannot hold the models above.
Price may vary. See full details → · Alternative: the ZOTAC Twin Edge OC 12GB is the same GPU in a two-slot cooler.
6 cores / 12 threads · Radeon integrated graphics · 65 W
Pros: Integrated graphics drive the display so the discrete card is 100% dedicated to inference and encode; 65 W default TDP is the lowest in this list; AM4 keeps the platform cost low; six Zen 3 cores are ample for a media server's housekeeping.
Cons: Fewer cores than the 5800X for CPU-side transcode fallback; the iGPU's video engine is not a substitute for NVENC on a busy library.
Per AMD's product page this is a six-core Zen 3 APU with Radeon graphics at a 65 W default TDP, and the integrated graphics are the entire reason it is here. On a box where the discrete card is doing two jobs, handing display output to the iGPU returns several hundred megabytes of VRAM and removes desktop compositing from the card's workload — free headroom for the KV cache at no additional cost.
The idle-power argument matters more than it looks. An always-on machine spends most of its life idle, and over 8,760 hours a year the idle draw dominates the electricity bill far more than the occasional inference burst does. A 65 W part on a modest board is the right starting point for that duty cycle.
Price may vary. See full details →
8 cores / 16 threads · 105 W · DDR4-3200 dual channel
Pros: Eight Zen 3 cores give real CPU-side transcode fallback when NVENC sessions are saturated or a codec is unsupported; the strongest prompt-processing headroom on AM4 at this price; unified 32 MB L3.
Cons: 105 W and no bundled cooler; adds nothing to token generation once the model is GPU-resident.
Per AMD this is an 8-core/16-thread part with a 105 W default TDP and DDR4 support up to 3200 MT/s. On a hybrid box it earns its slot in two specific situations: software transcoding when the encoder is busy or the source codec is not hardware-supported, and prompt processing when a model is too large to stay fully resident and layers spill to system RAM.
Be clear about what it does not buy. Two channels of DDR4-3200 is 51.2 GB/s of theoretical bandwidth against the card's ~360 GB/s, so offloaded token generation is bandwidth-bound and barely moves with core count — the public CPU benchmark set in the llamafile discussion shows a 16-core Zen 3 chip and a 12-core Zen 2 chip generating at 8.50 and 8.45 tok/s respectively on the same 7B Q6_K run. Buy this for transcode fallback and prefill, not for tokens per second.
Price may vary. See full details →
158 mm tower · single 120 mm fan · no pump
Pros: Nothing in it fails on a five-year duty cycle; sustained capacity for a 105 W chip under hours-long load; low noise floor for a machine in a living space; fits cases an AIO would not.
Cons: Not the top of any peak-thermal chart; single-tower capacity is the ceiling if you plan to push well past 105 W.
A hybrid box is under load in long stretches — a prompt-eval run, a two-hour transcode — and a cooler that handles bursts is not the same as one that handles duty cycles. The argument for a 158 mm air tower on an always-on machine is the absence of a pump: a closed loop adds a wear component that will fail before the CPU does, on a machine you would rather not open.
Noise is the other half. This box lives where the media it serves is watched, and a tower cooler running slowly at a steady thermal load is the quietest usable option at this price. A side-by-side against two closed-loop coolers on the same duty cycle lives in Noctua NH-U12S vs Corsair H150i vs Kraken M22 on a 24/7 Ryzen 7 5800X host.
Price may vary. See full details →
1 TB · SATA 6 Gb/s · up to 540 MB/s sequential read
Pros: Cheapest per gigabyte of any sensible model-library storage; sequential read is the only metric that matters for loading weights; leaves the NVMe slot free for media or a cache.
Cons: Not the drive for constant model-swapping or a large retrieval index; SATA cabling in a small case is fussier than M.2.
Model weights are a sequential-read workload that happens once per load and then never again while the model stays resident. At the listed 540 MB/s, a 14.5 GB quantized model pages in around 27 seconds; a mid-range NVMe drive at roughly 5,000 MB/s does it in about three. That is a one-off difference of under half a minute on a machine that stays up for weeks.
Where the cheap drive is genuinely wrong: if you swap models several times an hour, or run retrieval over a large document index on the same drive, random I/O becomes the workload and SATA shows it. For a box holding four or five quantized models and a media library, this is the correct place to not spend money.
Price may vary. See full details →
This is the one specification that decides whether the build works. Weights and KV cache are sized by your model choice; encode sessions claim frame buffers on top. The card must have room for all three simultaneously at your worst-case load, not at idle.
NVIDIA's support matrix documents the encoder generation and chip count per card; the cap on concurrent sessions on GeForce parts has been a driver-side limit that NVIDIA has revised over time. Plan around your measured stream count and verify against the matrix for your driver rather than a number from a forum post three years old.
A box that never powers down is judged on watts at idle. The 65 W Ryzen 5 5600G and a 170 W card that is idle most of the day beat a higher-TDP platform on annual cost, even if the peak figures say otherwise.
Keep them separate. Model weights want sequential read on cheap capacity; a media library wants bulk storage; a retrieval index wants random I/O. Buying one fast drive to do all three costs more and serves each worse.
Unlike a rack machine, this box lives in a room you use. A tower air cooler and a two-fan card at a steady load are the quiet configuration; a blower card and a pump are not.
Weights are derived from parameter count at ~4.85 bits each for Q4_K_M; the measured VRAM figures in the notes column come from llmrun.dev and Hardware Corner.
| Model size | Weights (Q4_K_M) | KV at 8K (fp16) | Total resident | Free on a 12 GB card | Measured note |
|---|---|---|---|---|---|
| 4B | ~2.4 GB | ~1.1 GB | ~3.5 GB | ~8.5 GB | Ample room for concurrent encode |
| 8B | ~4.8 GB | ~1.2 GB | ~6.0 GB | ~6.0 GB | 6.0 GB measured at 4K, 55.20 tok/s |
| 12B | ~7.3 GB | ~1.3 GB | ~8.6 GB | ~3.4 GB | 8.1 GB measured, 29.00 tok/s |
| 14B | ~8.5 GB | ~1.3 GB | ~9.8 GB | ~2.2 GB | 9.5 GB measured, 24.60 tok/s |
Read the "free" column as your encode budget. An 8B model leaves roughly six gigabytes of headroom and absorbs concurrent transcodes comfortably; a 14B model at a long context leaves so little that a burst of simultaneous streams is what triggers an allocation failure — usually at the worst moment, in the middle of an agent run.
Almost everyone over-buys here, because they size the machine for the worst evening they can imagine rather than the one they actually have.
Open your server's dashboard and look at a week of playback sessions. Count how many were direct play — the client took the original file untouched — versus how many actually invoked the encoder. On a library that has been curated for the devices in the house, the transcode count is frequently near zero, and a machine sized for four concurrent 4K transcodes spent money solving a problem that does not exist.
Two changes usually beat hardware: store a compatible codec profile for the clients you own, and cap remote-stream quality so a phone on cellular does not trigger a 4K transcode. Do those first, measure again, and then size the card to whatever transcoding is genuinely left. The VRAM contention analysis for this exact pairing goes deeper on where the two workloads actually collide.
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
— Mike Perry
Amazon & eBay listings, plus full specs and alternatives on each product page.
As an Amazon Associate, SpecPicks earns from qualifying purchases; we also earn on qualifying eBay purchases via the eBay Partner Network. Prices shown were last tracked at crawl time and may vary — check the listing for the current price.
Friendly Fire: AMD Ryzen 7 5800X CPU Review & Benchmarks vs. 5600X & 5900X — Gamers Nexus on YouTube