Local LLMs on the Ryzen 5 5600G: llama.cpp CPU Inference Numbers

The AMD Ryzen 5 5600G runs 7B-class local LLMs on CPU-only at roughly 6–8 tokens/second on Q4_K_M with dual-channel DDR4-3600 — usable for background summarization, agent-style batching, and interactive chat with a bit of patience, but not fast enough for real-time conversation. RAM bandwidth is the bottleneck, not the six Zen 3 cores. Add a used 12 GB RTX 3060 and you go from 6 tok/s to 65+.

The CPU-only inference audience and when the 5600G fits

There's a specific type of local-LLM builder who benefits from a CPU-only starting point: someone who wants to try running a 7B model on the box they already own before spending on a discrete GPU. The AMD Ryzen 5 5600G is the perfect chip for that experiment — six Zen 3 cores at 3.9 GHz base, 16 MB of L3, a Vega 7 iGPU that displaces the need for a discrete card for boot/display, and a 65 W TDP that runs on any cheap AM4 board with a stock cooler. Used in 2026 it's under $120.

The catch is that CPU-only inference has fundamentally different perf characteristics from GPU. The bottleneck isn't compute — Zen 3 has plenty of FP32 throughput — it's memory bandwidth. Every generation step reads the entire model weights across memory. On a 3060 that's 360 GB/s of GDDR6. On the 5600G that's ~48 GB/s of dual-channel DDR4-3600. The ratio is 7.5×; the token/s ratio ends up close to that.

That's the shape of the answer. If your workflow is short interactive chat where you want to see the answer typed at reading speed, 6–8 tok/s (about 4–5 words per second) is just fast enough. If your workflow is code completion, long-form generation, or agentic loops with many tool calls, CPU-only will feel painful and you should budget for a used RTX 3060 12GB.

Key takeaways

• Ryzen 5 5600G hits ~6–8 tok/s on a 7B Q4_K_M model, CPU-only, with DDR4-3600.
• RAM bandwidth is the ceiling — faster DDR4 kits (3600 vs 3200) show a measurable 10–15% gen speedup.
• Six Zen 3 cores are enough; llama.cpp scales linearly to ~8 threads, not much beyond.
• A 3B model runs interactive-fast (25–35 tok/s). A 13B model gets uncomfortably slow (2.5–4 tok/s).
• Adding a used RTX 3060 12GB is the single biggest upgrade — 10× throughput for ~$300.

How llama.cpp uses the 5600G's six Zen 3 cores

llama.cpp's CPU backend uses OpenMP threading with a per-tensor-op parallelization. For matrix multiplication in the transformer's attention and MLP layers, it splits the weight matrix along rows and dispatches to threads. On a 6-core / 12-thread chip like the 5600G, empirically the sweet spot is -t 6 (physical cores only) — going to -t 12 (SMT) actually slows throughput because the pairs of SMT threads contend for L1D and the Zen 3 load/store unit.

Concretely, on a 5600G at DDR4-3600 CL18 running Qwen2.5-7B-Instruct Q4_K_M:

• -t 4: ~5.2 tok/s
• -t 6: ~7.4 tok/s
• -t 8: ~7.3 tok/s (5600G only has 6 physical cores, extras go SMT)
• -t 12: ~6.1 tok/s (SMT contention)

The takeaway: match -t to physical cores. On the 5600G that's -t 6.

Does RAM speed matter more than core count for token throughput?

Yes. This is the counterintuitive-but-consistent finding on CPU-only inference for the last two years, and it holds on the 5600G. If you have to choose between spending on faster RAM or on a chip with more cores (in the same socket), take the faster RAM.

Rough numbers on the 5600G with the same 7B Q4_K_M model and -t 6:

• DDR4-2666 CL19: ~5.1 tok/s
• DDR4-3200 CL16: ~6.4 tok/s
• DDR4-3600 CL18: ~7.4 tok/s (sweet spot for AM4)
• DDR4-4000 CL18: ~7.6 tok/s (limited gain — 5600G IMC caps out)

Two things to note. First, the 5600G's Infinity Fabric prefers DDR4-3600 as the practical sweet spot; beyond that you drop out of 1:1 FCLK mode and gains taper. Second, tight timings matter — a CL14 kit at 3200 will outrun a CL20 kit at 3600 in some workloads. For local LLM specifically, timings matter less than raw bandwidth, so prioritize speed.

Spec table: 5600G vs the RAM bandwidth tiers

The 5700X row is worth calling out — it has more cores and more L3, but the token/s gain over a well-tuned 5600G is small (~12%). CPU inference is memory-bound; adding cores past the memory-bandwidth ceiling has diminishing returns.

Benchmark table: tok/s across 3B/7B/8B/13B on the 5600G (community measurements)

The clean break is around 8B parameters. Below that, CPU-only feels responsive. Above that, patience.

Quantization matrix: Qwen2.5-7B on the Ryzen 5 5600G

On a CPU-only rig, Q4_K_M is the pragmatic default — meaningful quality, tolerable speed. Q3_K_M is a fallback if you need a bit more responsiveness on a longer context.

Prefill vs generation: why long prompts hurt on CPU

Prefill scales with prompt length × model size × compute. On the 5600G with a 7B Q4_K_M model:

• 512-token prompt: ~2.5 seconds prefill
• 2K-token prompt: ~10 seconds prefill
• 4K-token prompt: ~22 seconds prefill
• 8K-token prompt: ~50 seconds prefill

That 50-second wait before you see the first token on an 8K RAG workload is the point where the 5600G stops being fun. On a 3060 the same prefill is under 4 seconds. If you plan on long-context retrieval workflows, the GPU upgrade is basically a requirement.

Context-length impact on RAM headroom

7B Q4_K_M weights are ~4.3 GB; KV cache at 8K context is ~2 GB (fp16). At 32K context, KV climbs to ~8 GB. On a 32 GB DDR4 system you have plenty of headroom — the real cost of longer context is prefill time, not memory. If you're on 16 GB total, cap context at 4K for a 7B model.

When to add a cheap RTX 3060 12GB instead of pushing the CPU

The math is unambiguous. A used ZOTAC RTX 3060 12GB or Gigabyte RTX 3060 Gaming OC at ~$280–320 takes a 5600G box from 7 tok/s to 65+ tok/s on the same 7B Q4_K_M model. That is a 9× throughput multiplier for roughly the same money as the CPU. The 5600G doesn't stop being useful — it drives the display, hosts the OS, and handles the tool-call driver code — but the LLM lives on the GPU.

Perf-per-dollar: APU-only vs APU + budget GPU

• APU-only rig (5600G + 32 GB DDR4-3600): ~$300 all-in, ~7 tok/s on 7B. That's ~$43 per (tok/s).
• APU + budget GPU (5600G + RTX 3060 12GB + 32 GB): ~$600 all-in, ~65 tok/s. That's ~$9 per (tok/s).

The GPU-added rig is nearly 5× better on the throughput-per-dollar axis. If your goal is doing meaningful local LLM work daily, the GPU is the correct spend.

Bottom line

The Ryzen 5 5600G is a legitimate entry-point for local LLM inference: it runs 7B models on CPU at a usable-but-slow ~7 tok/s, it runs 3B models genuinely fast, and it costs almost nothing used. It is not a long-term answer for anyone running LLMs daily — the throughput-per-dollar economics tilt hard toward adding a 12 GB RTX 3060 the moment you know the workflow will stick. Buy the 5600G if you're experimenting; upgrade to a 3060 the day you find yourself waiting for it.

Common CPU-only inference pitfalls

1. Using all 12 threads. Set -t 6 on the 5600G, not -t 12. SMT contention on Zen 3 hurts memory-bound workloads more than it helps.
2. Running at DDR4-2666 defaults. Enable XMP/DOCP on your board and get to 3600 CL18. This is a free 20% throughput uplift.
3. Loading full-precision weights. F16 on a 7B model won't fit in RAM comfortably alongside the OS. Q4_K_M is the pragmatic default.
4. Forgetting NUMA on multi-DIMM setups. Populate DIMMs symmetrically so dual-channel activates. A single-DIMM 32 GB config runs single-channel and halves bandwidth.
5. Underestimating prefill. 8K context prompts on CPU-only take 40–60 seconds to prefill. If your workflow is agent-style with lots of tool calls and long system prompts, add a GPU.

When NOT to buy the 5600G for local LLM work

• You already have a GPU with 8+ GB VRAM. Any modern GPU beats the 5600G on inference throughput. The 5600G's value case rests on the iGPU covering the display, not on CPU inference being fast.
• Your workflow needs sub-second first-token latency. CPU-only prefill is measured in seconds. Interactive chatbot use is fine; interactive code-completion is not.
• You process long-context RAG on lots of documents. Prefill scales linearly with input length; CPU-only hits a wall around 8K.
• You need real-time streaming into a UI. 7 tok/s on 7B is below typing speed. Users will notice.

Real-world example: a 5600G as a "starter LLM box"

Someone new to local inference in 2026:

• Buy a used Ryzen 5 5600G for ~$120.
• 32 GB DDR4-3600 kit for ~$75.
• A cheap B550 or B450 board for ~$85.
• 1 TB NVMe for ~$85.
• Case + PSU for ~$120.
• Total: ~$485 for a working PC that runs Qwen2.5-3B at 25 tok/s and Qwen2.5-7B at 7 tok/s.
• Six months later, add a used ZOTAC RTX 3060 12GB for ~$280. Now you're at 65 tok/s on the 7B model and 25 tok/s on 14B distills.

That upgrade path — CPU-first, GPU-later — is the single strongest argument for the 5600G in 2026.

Related guides

• Ryzen 7 5800X vs Ryzen 5 5600G for 1080p Gaming and Light Local LLM Work
• Best GPU for Local LLMs Under $300: The 12GB RTX 3060 Case
• Per-Model GPU VRAM Requirements for Local LLMs in 2026
• MSI RTX 3060 Ventus vs ZOTAC RTX 3060 Twin Edge: Which 12GB Card to Buy
• Ollama vs llama.cpp on the RTX 3060 12GB: Which Local LLM Runner Wins in 2026?

Citations and sources

• llama.cpp — CPU inference benchmarks (r/LocalLLaMA community reports)
• AMD — Ryzen 5 5600G product page
• Anandtech — Zen 3 memory subsystem analysis