Qwen3 4B on Raspberry Pi 4 8GB vs Ryzen 5 5600G: Which Cheap Always-On Box Wins?

The always-on-box reader

This comparison is for someone who wants a small language model answering around the clock: parsing Home Assistant voice intents, tagging incoming documents, summarizing RSS items, or backing a private chat endpoint for a household. You don't need a frontier model for that. You need something that is always on, costs little to run, and responds fast enough that the automation doesn't feel broken.

Qwen3 4B is a natural fit for that job. It has 4,022,468,096 parameters per the Hugging Face model index, and Qwen's official GGUF repository ships it at 2.50 GB in Q4_K_M. It also has a 40,960-token maximum context per its config.json. It's small enough for an 8 GB single-board computer and capable enough to be worth running.

Two candidates come up again and again for this job. One is the Raspberry Pi 4 Model B 8GB, which many readers already have in a drawer. The other is the AMD Ryzen 5 5600G, a desktop APU with six Zen 3 cores and integrated Radeon graphics, so it boots without a graphics card. Per AMD, it's rated 65 W TDP with dual-channel DDR4 support up to 3200 MT/s.

These two are closer on price than they are on performance. They also differ in ways that matter for a 24/7 deployment: thermals, storage, and idle power.

This piece is editorial synthesis of public measurements from Jeff Geerling's ai-benchmarks project, llama.cpp CPU results published by TechHara and in a ROCm issue thread, vendor datasheets from Raspberry Pi and AMD, and Qwen's own model files. No Pi 4 or 5600G result for Qwen3 4B specifically has been published by these sources, so the headline figures are scaled from the closest measured neighbours. The scaling is shown and labelled.

Key takeaways

• Qwen3 4B at Q4_K_M is a 2.50 GB file, and Q8_0 is 4.28 GB, per Qwen's GGUF repository. Every useful quant fits in 8 GB of RAM.
• BCM2711-class hardware generates 1.60 tok/s on Llama 3.2 3B at 6 W peak, measured on a Pi 400 in Geerling's results. That model's Q4_K_M file is 1.87 GiB per Geerling's llama.cpp output, which puts Qwen3 4B at roughly 1.3 tok/s on a Pi 4.
• Zen 3 APUs measure about 10 tok/s on a 7B Q4_0 model (TechHara, Ryzen 5 5600H) and 8.8 tok/s on Mistral 7B (ROCm issue #2774, Ryzen 5700G). Both scale to roughly 15 tok/s for Qwen3 4B.
• The integrated Radeon doubles prefill but not generation. TechHara measured pp512 rising from ~34 to ~76 tok/s on Vega 7 while generation stayed at ~10 tok/s (TechHara).
• The Pi 4's microSD slot peaks at 50 MB/s per the Pi 4 datasheet, which is why an SSD on its USB 3.0 ports matters.
• Energy per token comes out nearly even. At ~1.3 tok/s and 6 W versus ~15 tok/s and 65 W, both boxes spend roughly 1.2-1.3 kWh per million tokens (arithmetic below).

Step 0: what are you actually running on this box?

Decide on your latency floor before you choose hardware. The three common always-on workloads have very different floors.

Voice-intent routing (Home Assistant, a wake-word pipeline) produces short outputs, maybe 10-30 tokens of JSON or a single intent name, but somebody is standing in the kitchen waiting. At 1.3 tok/s, a 20-token response takes about 15 seconds before prompt processing is even counted. At 15 tok/s it's under 2 seconds. This workload rules out the Pi.

RAG chunk summarization and document tagging run unattended on a queue. Nobody waits on any single item, so throughput per day matters more than latency. A Pi 4 generating 1.3 tok/s continuously produces about 112,000 tokens a day. That's enough to tag hundreds of documents if each output is short, and not enough if each needs a paragraph.

Chat has a human reading along, and at conversational pace any figure under reading speed feels broken. The Pi 4 lands well below that. The 5600G is usable.

If your workload is the second category and the Pi is already deployed, the Pi is a defensible answer. For the other two, keep reading for the 5600G.

What Qwen3 4B needs: quantization and memory

File sizes are from Qwen's official GGUF repository, with Q3_K_M from Unsloth's Qwen3-4B-GGUF. The KV cache is derived from the published config.json: 36 layers, 8 key-value heads, and a head dimension of 128. That comes to 2 × 36 × 8 × 128 × 2 bytes = 147,456 bytes per token at FP16, or about 0.60 GB at 4,096 tokens. Bits per weight is llama.cpp's figure for each format, from the llama-quantize README. It's measured on a different model but describes the format itself.

Everything fits. On memory-bandwidth-bound hardware, though, file size is speed: every token reads the full weight set. Q8_0 is 1.7× the size of Q4_K_M, so on either machine it generates at roughly 60% of the Q4_K_M rate. On the Pi, that means going from about 1.3 tok/s to about 0.8 tok/s. Stay on Q4_K_M unless you've confirmed a quality problem for your specific task. Qwen doesn't publish per-quant accuracy for this model, so check it against your own prompts.

The 4 GB Pi 4 is a different story. Q4_K_M plus a 4K cache plus Raspberry Pi OS leaves very little margin, and once the board starts swapping to storage, generation stalls rather than slowing gracefully.

Spec delta: Raspberry Pi 4 8GB vs Ryzen 5 5600G

The memory row is the one that matters most. Both machines nominally run 3200-class memory. The difference in measured tok/s (1.60 on a 3B model for BCM2711 against roughly 10 on a 7B model for Zen 3) shows how much more usable bandwidth and compute the desktop platform delivers.

The Pi path in practice: the bare board and the Vilros 8GB kit

There are two ways to buy into the Pi path: the Raspberry Pi 4 Model B 8GB board on its own, or the Vilros Raspberry Pi 4 8GB Basic Starter Kit. The kit's listing describes a "Fan Cooled Heavy Duty Aluminum Alloy Case".

Cooling matters more for LLM inference than for almost any other Pi workload. A 4B model keeps all four Cortex-A72 cores busy for the entire generation, and at 1.3 tok/s a 200-token answer takes about two and a half minutes of sustained full load. The Pi 4 datasheet says the firmware governor "will throttle back both the CPU speed and voltage to make sure the CPU temperature never exceeds 85 degrees C". A bare board in still air will reach that limit during long generations, and the tok/s figure falls further from there.

So the kit isn't just packaging. A fan-cooled case keeps sustained throughput close to the benchmark figure. If you already own a bare board, a heatsink case with a small fan does the same job.

Power is the second practical issue. The datasheet specifies a supply "capable of delivering 5V at 3A". An underpowered phone charger causes brownouts under sustained load, and those look like software crashes.

To bound the Pi 4's speed, the measured data points in the BCM2711 family and its successor are:

Scaling the Pi 400 figure by weight size (1.87 GiB for Llama 3.2 3B Q4_K_M, 2.33 GiB for Qwen3 4B Q4_K_M) gives about 1.3 tok/s for Qwen3 4B on a Pi 4 8GB. Note that a Pi 5 would roughly triple that. If you're buying new specifically for this job, the Pi 4 isn't the board to buy. See best Raspberry Pi for a local LLM home lab.

The desktop APU path: the Ryzen 5 5600G

The AMD Ryzen 5 5600G brings six Zen 3 cores and twelve threads at 65 W, with dual-channel DDR4 support, per AMD. It has integrated Radeon graphics, so it needs no graphics card to boot.

Two independent public measurements on closely related Zen 3 APUs give a consistent picture. A Ryzen 5 5600H (6 cores, 12 threads, Zen 3) generated ~10 tok/s on Llama 2 7B Q4_0, a 3.56 GiB file, in TechHara's llama.cpp test. A Ryzen 5700G delivered "8.8 tokens/second with Mistral 7B" running CPU-only, per ROCm issue #2774.

Scaled by weight size to Qwen3 4B's 2.33 GiB Q4_K_M, both land at about 15 tok/s. The theoretical ceiling is about 51.2 GB/s of dual-channel DDR4-3200 bandwidth divided by 2.50 GB, or roughly 20 tok/s. The estimate sits sensibly below that line.

On x86, llama.cpp uses AVX2 kernels, and on the Pi it uses Arm NEON. The instruction set is part of the gap, but the larger part is simply more memory throughput and more capable cores.

About the Vega iGPU: it helps prefill and doesn't help generation. With all layers on the Vega 7 via Vulkan, TechHara saw prompt processing roughly double from ~34 to ~76 tok/s, while token generation "remained nearly identical (~10 t/s)" (TechHara). That result makes sense: the iGPU shares the same system memory bus, so it hits the same bandwidth ceiling. Use it for long prompts and to leave CPU cores free for other services.

Prefill vs generation on both boxes

A single blended "tok/s" figure hides the Pi's biggest weakness.

On the 5600G, TechHara's Zen 3 figures give ~34 tok/s CPU prefill on a 6.74B model and ~76 tok/s on the iGPU (TechHara). Prefill is compute-bound and scales roughly with parameter count, so for the 4.02B Qwen3 4B that works out to roughly 55-60 tok/s on the CPU and 120-130 tok/s on the iGPU. Both are estimates.

On the Pi 4, none of the sources used here publish a CPU prefill figure. On Zen 3, prefill runs about 3.4× generation (34 vs 10), and if the Pi keeps a similar ratio, it prefills Qwen3 4B at roughly 4-5 tok/s. Treat that as an order-of-magnitude estimate.

The Pi's 1.3 tok/s generation is tolerable for short outputs. Its prefill is what disqualifies it for anything with a system prompt. Home Assistant's conversation agent sends device context with every request, so a 300-token prompt is realistic, and a full minute of prefill before the first token is a broken experience.

Storage is not a footnote: the Kingston A400 960GB

The Pi 4 datasheet states the SD slot runs in "DDR50 mode (at a peak bandwidth of 50 Megabytes / sec)" (Pi 4 datasheet). Reading the 2.50 GB Qwen3 4B file at that ceiling takes at least 50 seconds, and more in practice, because real cards rarely sustain the peak. That penalty lands on every service restart that drops the page cache.

The board's "2 × USB 3.0 ports" (product brief) are the fix. A SATA SSD such as the Kingston A400 960GB in a USB 3.0 enclosure removes the SD bottleneck. Booting from it also takes the constant log and cache writes of an always-on host off the microSD card, which is the part most likely to wear out in a 24/7 deployment. Storage speed doesn't change steady-state tok/s, which is set by memory, but it decides how fast the box recovers after a reboot and how long it keeps running.

On the 5600G, the same drive connects over native SATA, and a 960 GB library holds a wide set of models. For more on this trade-off, see best storage for a Raspberry Pi 4 home server.

When neither is enough: the escape hatch to a 12GB GPU

If Qwen3 4B turns out to be too small, and 8B or 14B becomes the target, both boxes hit a wall. The Pi's 8 GB of soldered RAM is fixed. The 5600G can take more DDR4, but generation speed falls as file size grows: roughly 7 tok/s at 8B and 4 tok/s at 14B by the same scaling.

The escape hatch is a discrete card in the 5600G's PCIe slot. The MSI Gaming GeForce RTX 3060 12GB runs Qwen3 8B at 55.2 tok/s and Qwen3 14B at 31.2 tok/s at 4K context, per Hardware Corner. On 3B-class models, Geerling measured 122.85 tok/s at 214 W (Geerling #40).

This is the strongest structural argument for the 5600G. It's the only one of the two that can grow into a GPU box later without replacing anything. See Qwen3 8B: Raspberry Pi 4 8GB vs RTX 3060 12GB for that next step.

Performance per dollar and per watt for a 24/7 box

Electricity figures assume $0.15/kWh. Substitute your own tariff. The Pi figure uses the Pi 400's measured 6 W peak during inference (Geerling). The 5600G figure uses its 65 W TDP (AMD) as a package-only bound. The rest of that desktop draws power too.

Two things stand out. First, energy per token is almost identical between the Pi and the 5600G. The Pi draws a tenth of the power and does about a tenth of the work. The Pi isn't more efficient. It's just slower and cheaper to leave idle.

Second, a year at full load is a ceiling. Neither box runs flat out 24 hours a day on a household workload, and at idle the Pi's advantage is real. A desktop's idle floor depends on the motherboard, PSU and fans far more than on the 65 W CPU rating, so measure it at the wall before assuming.

On purchase price, the Pi board alone and the 5600G alone cost nearly the same. The 5600G still needs a motherboard, RAM, a PSU and a case. The Pi needs a supply, a case and storage.

Verdict matrix

Get the Raspberry Pi 4 8GB if you already own one, the workload is unattended queue processing (tagging, overnight summaries), outputs are short, and nobody waits on a response. About 112,000 tokens a day at a 6 W peak is useful capacity for that job. Counter-case: any workload with a system prompt, such as Home Assistant's conversation agent, means a minute or more of prefill per request. That's a real disqualifier, not a minor annoyance.

Get the Ryzen 5 5600G if anyone waits on the output, prompts carry context, or you think you might want 8B or 14B later. About 15 tok/s generation and single-digit-second prefill make Qwen3 4B genuinely interactive, and the empty PCIe slot is your upgrade path. Counter-case: if the building is power-constrained or the box must be silent and fanless, a desktop platform is the wrong form factor. Look at a Pi 5 instead.

Skip both and buy a 12GB GPU if 4B isn't really the target. If you're already sure you want 8B-14B-class answers, the RTX 3060 12GB is the component doing the work (Hardware Corner), and the host CPU barely matters. Counter-case: a GPU box's idle draw is the highest of the three options, so for a mostly idle household endpoint, it's more machine than the job needs.

The bottom line

For the default always-on reader running Qwen3 4B, buy the Ryzen 5 5600G. Scaled from two independent Zen 3 measurements (TechHara, ROCm #2774), it's roughly ten times faster than a Pi 4 at generation and far faster at prefill, and it uses about the same energy per token. It can also take a GPU later. Keep the Pi 4 you already own for the unattended jobs where 1.3 tok/s is enough, and don't buy a new Pi 4 for this workload.

Frequently asked questions

How much RAM does Qwen3 4B need at Q4_K_M?

The Q4_K_M file is 2.50 GB per Qwen's GGUF repository. A 4K-token FP16 KV cache adds about 0.60 GB, based on the model's 36-layer, 8-KV-head config. That's roughly 3.1 GB before the operating system. An 8 GB Pi 4 handles that comfortably. The 4 GB board technically loads it but leaves little room for context before it starts swapping, and swapping stalls generation.

Does the Ryzen 5 5600G's integrated graphics help with inference?

For prefill, yes. For generation, no. In TechHara's llama.cpp test on a closely related Zen 3 APU, moving layers to the Vega iGPU roughly doubled prompt processing from ~34 to ~76 tok/s, while generation stayed at about 10 tok/s (TechHara). The iGPU shares the same system memory bus, so it hits the same bandwidth ceiling during generation. Use it to shorten waits on long prompts.

Do I need active cooling on the Raspberry Pi for this?

Yes, if the box runs inference throughout the day. A 4B model keeps all four Cortex-A72 cores busy for minutes per answer. The Pi 4 datasheet says the firmware throttles CPU speed and voltage so the chip never exceeds 85°C, which cuts sustained throughput below benchmark figures. A fan-cooled case, like the one in the Vilros 8GB kit, keeps the board near its measured speed.

Can I skip the SSD and run everything from a microSD card?

You can, but you give up two things. The Pi 4's SD interface peaks at 50 MB/s per its datasheet, so loading the 2.50 GB model takes at least 50 seconds after every cache-clearing restart. More importantly, the constant log and cache writes of an always-on host wear microSD cards out. Booting from a SATA SSD on one of the two USB 3.0 ports fixes both problems.

At what point should I stop scaling CPUs and buy a GPU?

When the model grows past roughly 4B, or when users expect conversational speed. By the same weight-size scaling used here, a 5600G falls to about 7 tok/s at 8B and about 4 tok/s at 14B. An RTX 3060 12GB runs Qwen3 8B at 55.2 tok/s and Qwen3 14B at 31.2 tok/s per Hardware Corner. That makes 12 GB of VRAM the clean break point once a small model stops being enough.

Related guides

• Raspberry Pi 4 8GB vs Ryzen 5 5600G: which homelab box wins?
• Llama 3.2 3B: Raspberry Pi 4 8GB vs RTX 3060 12GB
• RTX 3060 12GB local LLM guide: which models actually fit
• Best parts for an always-on local LLM server
• Ryzen 5 5600G vs Ryzen 7 5800X for a 24/7 Ollama box

Live price comparison

Current listings for both sides: the Raspberry Pi 4 Model B 8GB and the AMD Ryzen 5 5600G. CPU-level benchmark data for the APU is at /benchmarks/amd-ryzen-5-5600g. Prices come from the retailer at the last refresh and may vary. Confirm at checkout.

As an Amazon Associate, SpecPicks earns from qualifying purchases.

Citations and sources

• Jeff Geerling, ai-benchmarks commit adding the Pi 400 result (accessed 2026-09-16)
• Jeff Geerling, ai-benchmarks issue #7: Raspberry Pi 5 CPU results (accessed 2026-09-16)
• Jeff Geerling, ai-benchmarks issue #40: RTX 3060 12GB llama.cpp results (accessed 2026-09-16)
• TechHara, llama.cpp benchmark: CPU vs iGPU on Ryzen 5 5600H (accessed 2026-09-16)
• ROCm issue #2774: APU support for 5600G/5700G, CPU-only llama.cpp result (accessed 2026-09-16)
• Hardware Corner, RTX 3060 12GB LLM benchmarks (accessed 2026-09-16)
• Hugging Face model index, Qwen/Qwen3-4B (accessed 2026-09-16)
• Qwen3-4B config.json (accessed 2026-09-16)
• Qwen, Qwen3-4B-GGUF (accessed 2026-09-16)
• Unsloth, Qwen3-4B-GGUF (accessed 2026-09-16)
• llama.cpp, llama-quantize README (accessed 2026-09-16)
• Raspberry Pi 4 Model B datasheet (accessed 2026-09-16)
• Raspberry Pi 4 Model B product brief (accessed 2026-09-16)
• Raspberry Pi 400 product brief (accessed 2026-09-16)
• AMD, Ryzen 5 5600G specifications (accessed 2026-09-16)

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.