A local-LLM rig running entirely on the GPU needs only enough CPU to feed the model — 6 cores handles 99% of chat workloads. Once you spill layers to system RAM, or you drive a long-context prompt through prefill, CPU cores start to matter. For pure inference on an RTX 3060 12GB, a Ryzen 5 5600G is enough. For CPU offload of 30B+ models or long batch prefill, an 8-core Ryzen 7 5800X or 5700X pulls ahead.
Why the CPU-cores question keeps coming up
The r/LocalLLaMA and r/homelab weekly threads are full of "should I get 6 cores or 12 cores for my LLM box" questions, and the honest answer is "it depends on where the model runs." Chat-model benchmarks assume everything fits in VRAM and generation is dominated by GPU bandwidth. That's the workload most people actually run. But three cases break that assumption:
- The model doesn't fit in VRAM — 30B+ on 12GB, or 70B on 24GB — and layers spill to CPU/RAM.
- The prompt is huge — a 16K-token repo dump for code review triggers a prefill pass that's memory-bandwidth limited.
- You're serving multiple concurrent conversations — each request runs its own tokenization and sampling loop.
Only case 1 is bandwidth-bound in a way cores can't fix. Cases 2 and 3 scale roughly linearly with core count up to about 8 cores, then flatten.
Key takeaways
- Pure-GPU inference: 6 cores is enough. Ryzen 5 5600G, Ryzen 5 5600, or Intel i5-12400 all handle it.
- CPU-offloaded inference (spilled layers): 8 cores meaningfully help. Ryzen 7 5800X or 5700X are the sweet spot.
- Long-context prefill: More cores + higher single-thread perf both help. 8 cores at 4.5+ GHz is the target.
- Multi-tenant serving: Each concurrent request consumes ~1 core for tokenization; 12+ cores start paying off.
- Never buy 16+ cores just for a solo home LLM. The next bottleneck will be memory bandwidth, not cores.
Where CPU cores actually get used in llama.cpp / Ollama
A single-user chat request through llama.cpp does five things:
- Tokenize the prompt — CPU, single-threaded per request.
- Prefill the KV cache — GPU if the model is GPU-resident, CPU if partially or wholly offloaded.
- Sample logits — CPU, single-threaded per request.
- Generate tokens — GPU-bound if resident; RAM-bandwidth-bound if offloaded.
- Detokenize + stream — CPU, negligible.
Steps 1, 3, and 5 barely register on a modern 6-core CPU. Step 2 dominates on the first prompt of a long conversation. Step 4 is the whole party during token streaming.
If your model fits in VRAM entirely, cores 1-3 sit idle. Adding cores 7-12 buys you nothing.
Benchmark table: Ryzen 5 5600G vs Ryzen 7 5800X on the same 3060 rig
Community measurements aggregated from public llama.cpp trackers. Configuration: RTX 3060 12GB, 32GB DDR4-3600 dual-channel G.SKILL RipJaws V, Ubuntu 24.04, llama.cpp Q4_K_M unless noted.
| Workload | Ryzen 5 5600G | Ryzen 7 5800X | Delta |
|---|---|---|---|
| Llama 3.1 8B Q4, 2K context, GPU-resident | ~55 tok/s | ~55 tok/s | 0% |
| Qwen 2.5 14B Q4, 2K context, GPU-resident | ~30 tok/s | ~30 tok/s | 0% |
| Prefill Llama 3.1 8B, 16K prompt | ~2.1s | ~1.4s | 33% |
| Qwen 2.5 32B Q3, 20 layers offloaded to CPU | ~2.1 tok/s | ~3.8 tok/s | 81% |
| Llama 3.1 70B Q4, 60 layers offloaded | ~0.3 tok/s | ~0.5 tok/s | 67% |
| Two concurrent 8B Q4 chat requests | ~30/28 tok/s | ~45/44 tok/s | 55% |
The read: when everything fits in VRAM, the CPU is invisible. The moment you offload or prefill, cores start earning their price.
How much RAM matters vs how many cores matter
Per AMD's public product page, the Ryzen 7 5800X supports DDR4-3200 in dual-channel — about 51 GB/s of memory bandwidth. That's the actual ceiling for CPU-offloaded inference on the AM4 platform. More cores can't push more bandwidth than the memory controller allows.
On a spilled-layer workload:
- 6 cores + dual-channel DDR4-3600: ~2.1 tok/s on 32B Q3 with 20 layers offloaded.
- 8 cores + dual-channel DDR4-3600: ~3.8 tok/s — cores are still the bottleneck at this thread count.
- 12 cores + dual-channel DDR4-3600: ~4.1 tok/s — cores stop helping, bandwidth is saturated.
- 16 cores + quad-channel DDR4-3200: ~7.2 tok/s (on Threadripper class) — extra memory channels open the ceiling back up.
For a consumer AM4/AM5 desktop with dual-channel memory, 8 cores is the practical ceiling. Beyond that you're waiting for DRAM.
Prefill-heavy workloads (long prompts) — where cores earn their keep
If you paste a 16K-token codebase into a code-review prompt, prefill runs on the GPU only if the entire model + KV cache is GPU-resident. On an RTX 3060 12GB, a 14B model at Q4 leaves roughly 5-6GB for KV, which caps context around 24-32K depending on model. Past that, KV spills to CPU RAM and prefill becomes a mixed CPU+GPU operation.
Community measurements on a 14B Q4 model with a 32K prompt:
| Configuration | Prefill time |
|---|---|
| 5600G + 3060, 24K context (fits) | ~2.8s |
| 5600G + 3060, 32K context (spills) | ~14s |
| 5800X + 3060, 32K context (spills) | ~9s |
The 8-core CPU cuts spilled prefill by ~35%. That's a real quality-of-life difference when you're iterating.
Multi-user / multi-request serving
A single LLM server (Ollama, vLLM, text-generation-inference) can batch multiple requests, but each has its own tokenization loop and per-request state management. With 4 concurrent chat sessions on a 6-core 5600G, you'll see the CPU utilization pin most cores at 60-80% during peak token generation. On an 8-core 5800X, the same load sits at 45-55%.
If your box is doing anything else while inference runs — Docker containers, a Home Assistant instance, Jellyfin transcoding — you want the extra cores. A dedicated inference-only box is fine with 6.
Recommended pairings by workload
Pure chat / code assistant, single user, model fits in VRAM
- CPU: Ryzen 5 5600G or Ryzen 5 5600
- Rationale: 6 cores is invisible on GPU-resident inference.
- Alternative: Intel i5-12400 for the same price.
Mixed chat + occasional 30B/70B offload
- CPU: Ryzen 7 5700X or Ryzen 7 5800X
- Rationale: 8 cores meaningfully helps offloaded inference.
- The 5700X trades 300MHz for 40W lower TDP — better for always-on boxes.
Long-context RAG or codebase review
- CPU: 8+ cores with high single-thread perf. Ryzen 7 5800X, Ryzen 7 7700, Intel i7-12700.
- RAM: 32GB DDR4-3600 dual-channel minimum. 64GB if you plan on 32K+ context on 30B+ models.
Multi-user home / small-team serving
- CPU: 12+ cores (Ryzen 9 5900X, Ryzen 7 7700X + workloads-only-mode).
- Motherboard: ASUS Prime X570-Pro or similar with PCIe 4.0 x16 for future GPU upgrades.
Common pitfalls
- Buying 16 cores for a solo LLM box. Wasted money — you'll hit memory bandwidth long before core count.
- Using single-channel RAM. Halves your CPU-offload throughput. Always populate two DIMM slots symmetrically.
- Pairing a strong CPU with a weak GPU. For consumer inference, GPU dominates. Spend the marginal dollar on VRAM before cores.
- Ignoring the APU option. The 5600G's integrated GPU is useless for LLM work but the 6 cores are enough — save the GPU purchase for one dedicated card.
- Assuming more cores = faster prefill always. True up to ~8 cores on consumer platforms; flat after that.
When NOT to overspec the CPU
If any of these are true, buy 6 cores and put the savings into a bigger GPU or more RAM:
- You never run models that spill to CPU.
- You only run one chat session at a time.
- You have a discrete GPU with enough VRAM for your target model.
- The rig has a specific role (inference only) and won't host other workloads.
Every extra core past 6 is roughly $30-50 in AM4/AM5 economy pricing. That money buys measurable extra VRAM if you spend it on the GPU line item instead.
Motherboard chipset compatibility notes
The AM4 platform (5600G, 5700X, 5800X, 5900X) is the honest sweet spot in 2026 — it hit end-of-life at the retail level but is still available in bulk on the used market and pairs with cheap DDR4. Recommended chipsets:
- B550 — best value for a 5600G or 5700X build. PCIe 4.0 x16 for the GPU, PCIe 4.0 M.2 for the boot drive, four SATA ports, USB 3.2 Gen2 on most boards. The ASUS Prime X570-Pro is a solid option here if you want an X570 for extra M.2 lanes.
- A520 — cheaper, works with Ryzen 5 and lower Ryzen 7. Drops PCIe 4.0 support, which mostly affects future GPU upgrades. Fine for a stable box.
- X570 — worth it only for multi-M.2 workloads. For a straight LLM box, the extra $50-80 is better spent on a bigger GPU or more RAM.
AM5 (7000/9000 series) is the future path but adds ~30-40% to the platform cost via mandatory DDR5. For LLM-only workloads, that premium doesn't pay off — the CPU work is either negligible (GPU-resident) or bandwidth-bound (offloaded), and DDR5 bandwidth advantage is smaller than the price gap suggests.
Power and thermal considerations
The Ryzen 5 5600G is a 65W-TDP part; the Ryzen 7 5800X is 105W. Neither needs an exotic cooler for pure inference — even the stock Wraith Prism (bundled with the 5800X's box variants) is adequate. Where thermal design starts to matter is on the Ryzen 7 5700X — its 65W spec is honest but sustained CPU offload can push it toward its thermal limit for hours, at which point a $50 tower cooler is a legitimate quality-of-life upgrade.
Under sustained LLM load with an offloaded 70B model, the 5800X will pull 90-100W on average; the 5600G tops out around 60W. Over a year of always-on inference, that's roughly $30-45 in electricity at $0.13/kWh. Real but not decisive.
What to skip when in doubt
Three things routinely appear in "future-proofing" first-build parts lists that are safe to skip on a dedicated LLM box:
- A high-end mainboard. For pure inference, VRM quality above the mid-tier B550/X570 minimum doesn't buy anything. The CPU never pulls close to its power limit on GPU-resident workloads.
- PCIe Gen 5 storage. LLM model files are loaded once at startup, then memory-mapped. A $50 Crucial BX500 SATA SSD is functionally identical to a $300 Gen 5 drive for this workload.
- DDR5 on an AM5 platform. DDR4 dual-channel on AM4 is roughly $180 for 32GB in 2026; DDR5 for the same capacity is $260-320. On an inference-only rig, the delta buys nothing measurable.
Spend the marginal dollars on VRAM or extra RAM capacity instead. Both move the throughput needle further than platform upgrades do.
Bottom line
Six cores is enough for the workload most local-LLM users actually run. Move to 8 cores when you know you'll offload or prefill regularly. Move past 8 cores only if you're serving multiple users or your box has a second job. The Ryzen 5 5600G is the honest recommendation for the majority; the Ryzen 7 5800X is the honest upgrade path for the offload case.
Sanity-check the CPU pick with these three questions
Before committing to a CPU tier for a local-LLM build, answer these three:
- Will the model always fit in VRAM? If yes, 6 cores is enough. If no, you'll offload — plan for 8 cores minimum.
- What's the longest prompt you'll routinely send? Under 4K tokens, prefill is trivial. Between 4K-16K, more cores help. Past 16K on a model that spills to CPU, you want the fastest 8-core you can afford.
- Will the box do anything else? A dedicated inference-only box is fine with 6. A Docker host + LLM + game streaming + occasional Steam-Deck-companion role justifies 8+.
Answer honestly. Half the "should I get more cores?" posts on r/LocalLLaMA reveal the answer isn't cores at all — it's that the poster hasn't yet decided which model tier they'll actually run.
Related guides
- AMD Ryzen AI Halo vs DIY RTX 3060 Local-LLM Rig
- Ryzen 5 5600G APU Local LLM Inference (2026)
- Dual-Channel RAM for Local LLM Inference
- Best GPU for Local LLM (2025)
Citations and sources
- AMD — Ryzen 7 5800X product page
- ggerganov/llama.cpp — README + performance notes
- Phoronix — Ryzen 5000 series memory bandwidth analysis
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
