As an Amazon Associate, SpecPicks earns from qualifying purchases. See the SpecPicks review methodology.
Best Parts for a Local LLM Box That Also Runs Jellyfin in 2026
By Mike Perry · Published 2026-09-17 · Last verified 2026-09-17 · 11 min read
Most "best parts for a local LLM" guides assume the machine does one thing. The reader on a budget rarely has that luxury — there is one always-on box under the desk, and it is expected to answer prompts, serve a media library, and not sound like a hair dryer while doing it.
That consolidation creates one genuine engineering problem and a lot of false ones. The real problem is VRAM contention: model weights, the KV cache and the encoder's frame buffers all live in the same memory, and only one of those three shrinks on demand. The false problems are the ones people spend money on — a faster CPU for "transcoding headroom" on a library that mostly direct-plays, or an NVMe drive to shave seconds off a model load that happens once a day.
This guide picks five parts against that reality, with the throughput figures cited rather than asserted. The overall winner is the part that solves the contention problem instead of papering over it.
| Pick | Best For | Key Spec | Price Range | Verdict |
|---|---|---|---|---|
| MSI RTX 3060 12GB | Best overall | 12 GB GDDR6, ~360 GB/s, 170 W | Mid | The only pick that solves VRAM contention |
| AMD Ryzen 5 5600G | Best value | 6C/12T, Radeon iGPU, 65 W | Low | iGPU display output frees the whole card |
| AMD Ryzen 7 5800X | Heavy transcoding | 8C/16T, 105 W | Low-mid | CPU fallback when NVENC is saturated |
| Noctua NH-U12S | Performance add-on | 158 mm tower, no pump | Low | Nothing to fail on a box that never sleeps |
| Crucial BX500 1TB | Budget pick | SATA, up to 540 MB/s | Low | Model library storage, not a bottleneck |
🏆 Best Overall: MSI Gaming GeForce RTX 3060 12GB
12 GB GDDR6 · 192-bit · ~360 GB/s · 170 W · 7th-gen NVENC
Pros: 12 GB holds a 14B model resident with room left over; a dedicated encoder block means transcoding does not steal shader time from inference; 170 W is modest for a 24/7 box; the widest public benchmark coverage of any budget inference card.
Cons: Ampere-era efficiency is behind current generations; the 8 GB variant shares the name and is not the same product; encoder and model compete for the same memory pool.
This is the pick because it is the only one that addresses the actual conflict. Per TechPowerUp the card carries 12 GB on a 192-bit bus for roughly 360 GB/s, and NVIDIA's support matrix lists it as a 7th-generation NVENC part with one encoder chip — the encode work runs on fixed-function silicon, not the shaders your model is using.
On the inference side, Hardware Corner's RTX 3060 12GB benchmarks measure an 8B model at 55.20 tok/s using 6.0 GB at 4K context and 42.00 tok/s using 7.5 GB at 16K. llmrun.dev's database adds 12B-class models at 29.00 tok/s using 8.1 GB and 14B at 24.60 tok/s using 9.5 GB. Those VRAM figures are the important column: an 8B model at a working context leaves four to six gigabytes free, which is the headroom concurrent encode sessions need.
Buy the 12 GB part specifically — the 8 GB RTX 3060 runs a 128-bit bus and cannot hold the models above.
Price may vary. See full details → · Alternative: the ZOTAC Twin Edge OC 12GB is the same GPU in a two-slot cooler.
💰 Best Value: AMD Ryzen 5 5600G
6 cores / 12 threads · Radeon integrated graphics · 65 W
Pros: Integrated graphics drive the display so the discrete card is 100% dedicated to inference and encode; 65 W default TDP is the lowest in this list; AM4 keeps the platform cost low; six Zen 3 cores are ample for a media server's housekeeping.
Cons: Fewer cores than the 5800X for CPU-side transcode fallback; the iGPU's video engine is not a substitute for NVENC on a busy library.
Per AMD's product page this is a six-core Zen 3 APU with Radeon graphics at a 65 W default TDP, and the integrated graphics are the entire reason it is here. On a box where the discrete card is doing two jobs, handing display output to the iGPU returns several hundred megabytes of VRAM and removes desktop compositing from the card's workload — free headroom for the KV cache at no additional cost.
The idle-power argument matters more than it looks. An always-on machine spends most of its life idle, and over 8,760 hours a year the idle draw dominates the electricity bill far more than the occasional inference burst does. A 65 W part on a modest board is the right starting point for that duty cycle.
Price may vary. See full details →
🎯 Best for Heavy Transcoding: AMD Ryzen 7 5800X
8 cores / 16 threads · 105 W · DDR4-3200 dual channel
Pros: Eight Zen 3 cores give real CPU-side transcode fallback when NVENC sessions are saturated or a codec is unsupported; the strongest prompt-processing headroom on AM4 at this price; unified 32 MB L3.
Cons: 105 W and no bundled cooler; adds nothing to token generation once the model is GPU-resident.
Per AMD this is an 8-core/16-thread part with a 105 W default TDP and DDR4 support up to 3200 MT/s. On a hybrid box it earns its slot in two specific situations: software transcoding when the encoder is busy or the source codec is not hardware-supported, and prompt processing when a model is too large to stay fully resident and layers spill to system RAM.
Be clear about what it does not buy. Two channels of DDR4-3200 is 51.2 GB/s of theoretical bandwidth against the card's ~360 GB/s, so offloaded token generation is bandwidth-bound and barely moves with core count — the public CPU benchmark set in the llamafile discussion shows a 16-core Zen 3 chip and a 12-core Zen 2 chip generating at 8.50 and 8.45 tok/s respectively on the same 7B Q6_K run. Buy this for transcode fallback and prefill, not for tokens per second.
Price may vary. See full details →
⚡ Best Performance Add-on: Noctua NH-U12S
158 mm tower · single 120 mm fan · no pump
Pros: Nothing in it fails on a five-year duty cycle; sustained capacity for a 105 W chip under hours-long load; low noise floor for a machine in a living space; fits cases an AIO would not.
Cons: Not the top of any peak-thermal chart; single-tower capacity is the ceiling if you plan to push well past 105 W.
A hybrid box is under load in long stretches — a prompt-eval run, a two-hour transcode — and a cooler that handles bursts is not the same as one that handles duty cycles. The argument for a 158 mm air tower on an always-on machine is the absence of a pump: a closed loop adds a wear component that will fail before the CPU does, on a machine you would rather not open.
Noise is the other half. This box lives where the media it serves is watched, and a tower cooler running slowly at a steady thermal load is the quietest usable option at this price. A side-by-side against two closed-loop coolers on the same duty cycle lives in Noctua NH-U12S vs Corsair H150i vs Kraken M22 on a 24/7 Ryzen 7 5800X host.
Price may vary. See full details →
🧪 Budget Pick: Crucial BX500 1TB SATA SSD
1 TB · SATA 6 Gb/s · up to 540 MB/s sequential read
Pros: Cheapest per gigabyte of any sensible model-library storage; sequential read is the only metric that matters for loading weights; leaves the NVMe slot free for media or a cache.
Cons: Not the drive for constant model-swapping or a large retrieval index; SATA cabling in a small case is fussier than M.2.
Model weights are a sequential-read workload that happens once per load and then never again while the model stays resident. At the listed 540 MB/s, a 14.5 GB quantized model pages in around 27 seconds; a mid-range NVMe drive at roughly 5,000 MB/s does it in about three. That is a one-off difference of under half a minute on a machine that stays up for weeks.
Where the cheap drive is genuinely wrong: if you swap models several times an hour, or run retrieval over a large document index on the same drive, random I/O becomes the workload and SATA shows it. For a box holding four or five quantized models and a media library, this is the correct place to not spend money.
Price may vary. See full details →
What to look for in a hybrid LLM and media server
VRAM budget after the encoder takes its slice
This is the one specification that decides whether the build works. Weights and KV cache are sized by your model choice; encode sessions claim frame buffers on top. The card must have room for all three simultaneously at your worst-case load, not at idle.
NVENC session limits are a driver property, not a silicon one
NVIDIA's support matrix documents the encoder generation and chip count per card; the cap on concurrent sessions on GeForce parts has been a driver-side limit that NVIDIA has revised over time. Plan around your measured stream count and verify against the matrix for your driver rather than a number from a forum post three years old.
Idle power, not peak power
A box that never powers down is judged on watts at idle. The 65 W Ryzen 5 5600G and a 170 W card that is idle most of the day beat a higher-TDP platform on annual cost, even if the peak figures say otherwise.
Storage split: weights versus media
Keep them separate. Model weights want sequential read on cheap capacity; a media library wants bulk storage; a retrieval index wants random I/O. Buying one fast drive to do all three costs more and serves each worse.
Noise, because you can hear this one
Unlike a rack machine, this box lives in a room you use. A tower air cooler and a two-fan card at a steady load are the quiet configuration; a blower card and a pump are not.
Table: VRAM budget under contention
Weights are derived from parameter count at ~4.85 bits each for Q4_K_M; the measured VRAM figures in the notes column come from llmrun.dev and Hardware Corner.
| Model size | Weights (Q4_K_M) | KV at 8K (fp16) | Total resident | Free on a 12 GB card | Measured note |
|---|---|---|---|---|---|
| 4B | ~2.4 GB | ~1.1 GB | ~3.5 GB | ~8.5 GB | Ample room for concurrent encode |
| 8B | ~4.8 GB | ~1.2 GB | ~6.0 GB | ~6.0 GB | 6.0 GB measured at 4K, 55.20 tok/s |
| 12B | ~7.3 GB | ~1.3 GB | ~8.6 GB | ~3.4 GB | 8.1 GB measured, 29.00 tok/s |
| 14B | ~8.5 GB | ~1.3 GB | ~9.8 GB | ~2.2 GB | 9.5 GB measured, 24.60 tok/s |
Read the "free" column as your encode budget. An 8B model leaves roughly six gigabytes of headroom and absorbs concurrent transcodes comfortably; a 14B model at a long context leaves so little that a burst of simultaneous streams is what triggers an allocation failure — usually at the worst moment, in the middle of an agent run.
The most-missed step: measure your real transcode load first
Almost everyone over-buys here, because they size the machine for the worst evening they can imagine rather than the one they actually have.
Open your server's dashboard and look at a week of playback sessions. Count how many were direct play — the client took the original file untouched — versus how many actually invoked the encoder. On a library that has been curated for the devices in the house, the transcode count is frequently near zero, and a machine sized for four concurrent 4K transcodes spent money solving a problem that does not exist.
Two changes usually beat hardware: store a compatible codec profile for the clients you own, and cap remote-stream quality so a phone on cellular does not trigger a 4K transcode. Do those first, measure again, and then size the card to whatever transcoding is genuinely left. The VRAM contention analysis for this exact pairing goes deeper on where the two workloads actually collide.
FAQ
Does video transcoding steal VRAM from the model I am running? Yes. Each active encode session reserves memory for frame buffers and lookahead, and that reservation comes out of the same 12 GB the weights and KV cache occupy. An 8B model leaves about six gigabytes free and absorbs a few concurrent 1080p sessions; a 14B model at long context leaves barely two and will fail an allocation under the same burst.
Should I use the Ryzen 5 5600G's integrated graphics for transcoding instead? Use them for display output. That is where they pay — freeing the discrete card entirely for inference and NVENC. The iGPU's video engine can handle light work, but it does not match a dedicated encoder block's throughput or codec coverage, so treat it as the reason to buy the chip rather than your transcode path.
Is a SATA SSD fast enough for a model library? Yes for storing weights. At 540 MB/s a 14.5 GB model loads in roughly 27 seconds against about three on a mid-range NVMe drive — a one-off cost that never recurs while the model stays resident. NVMe earns its price when you swap models constantly or run a retrieval index on the same drive.
How much power does an always-on box like this actually draw? Idle dominates, because idle is where the machine lives. A 65 W-class platform with a 170 W card sits far below its peak for most of the day and climbs only during inference or an encode burst. Over a year, the idle figure is what shows up on the bill.
When should I split this into two machines? When the two workloads collide on a schedule. If the household streams every evening at the same hour you want agent runs going, or your model needs the full 12 GB with no room for encode buffers, a second low-power box for media is cheaper than buying a larger card to absorb both.
Sources
- TechPowerUp — GeForce RTX 3060 12 GB specifications
- NVIDIA — Video encode and decode GPU support matrix
- AMD — Ryzen 5 5600G product page
- Hardware Corner — RTX 3060 12GB LLM benchmarks
- llmrun.dev — RTX 3060 12GB model database
Related guides
- Jellyfin NVENC Transcodes + a Local LLM on One RTX 3060 12GB
- Best Parts for an Always-On Local LLM Server in 2026
- Best GPU for a NAS or Homelab Running Local LLMs (2026)
- Best Parts for a Gaming PC That Also Runs Local LLMs in 2026
Citations and sources
- TechPowerUp — GeForce RTX 3060 12 GB specifications (accessed 2026-09-17)
- NVIDIA — Video encode and decode GPU support matrix (accessed 2026-09-17)
- AMD — Ryzen 5 5600G product page (accessed 2026-09-17)
- AMD — Ryzen 7 5800X product page (accessed 2026-09-17)
- Hardware Corner — RTX 3060 12GB LLM benchmarks (accessed 2026-09-17)
- llmrun.dev — RTX 3060 12GB model database (accessed 2026-09-17)
- mozilla-ai/llamafile — CPU benchmark discussion #450 (accessed 2026-09-17)
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
— Mike Perry · Last verified 2026-09-17
