A Raspberry Pi 4 8GB can run a local LLM, but you should set your expectations at ~1-3 tokens per second on a 3B model at q4 — enough for a slow chat bot or a self-hosted assistant that responds in sentences over a minute or two, not enough for anything interactive or agent-driven. Per LocalLLaMA community measurements posted throughout 2025-2026, that is the realistic bar for CPU-only ARMv8 inference at this hardware tier.
Editorial setup — why anyone still runs LLMs on a Pi 4
The reasons real people run llama.cpp on a Pi 4 8GB in 2026 have not changed: silent, always-on, sub-5-watt idle, network-attached, and cheap enough to dedicate to one task. The Pi 5 is faster on every axis, but the Pi 4 is still in service on thousands of home labs, and the 8 GB variant is the model with enough RAM to fit a 3B q4 model plus a KV cache plus the OS.
The question is not "should you buy a Pi 4 to run LLMs" — the answer is no, buy a Pi 5 or a $200 mini PC. The question is "if I already have a Pi 4 8GB, what LLM workloads make sense on it." That is what this article is about.
Key takeaways
- Realistic throughput: 1-3 tok/s on a 3B q4 model, single user, no other workload.
- Best model class: 1B-3B parameters at q4 (Phi-2, TinyLlama, Gemma 2B, Qwen 2.5 1.5B).
- CPU inference only. The VideoCore GPU is not viable for llama.cpp workloads.
- Storage: run the model off an SSD like the Crucial BX500 1TB over USB 3.0, not the microSD.
- Best fit: home-assistant intent classification, small summarization tasks, always-on side projects. Not for chat.
What the Raspberry Pi 4 8GB actually is
Per the Raspberry Pi Foundation's spec sheet, the Pi 4 8GB ships with a Broadcom BCM2711 SoC — quad-core Cortex-A72 at 1.5 GHz (overclockable to 2.0 GHz in most units), 8 GB of LPDDR4-3200 SDRAM, dual USB 3.0 ports, gigabit Ethernet, and dual micro-HDMI. Total power draw idles around 3 W and peaks near 7-8 W under heavy CPU load per community measurements.
For LLM work, the interesting numbers are: single-thread integer throughput on the A72 (baseline for llama.cpp CPU inference), memory bandwidth from the LPDDR4 subsystem (peak ~25 GB/s theoretical, ~10-15 GB/s sustained per benchmark reports), and lack of any NEON-accelerated matmul path beyond what llama.cpp already ships.
Benchmark table — Pi 4 8GB LLM throughput
Community measurements from r/LocalLLaMA and the llama.cpp project's benchmarks discussion, all Q4_K_M GGUF on a Raspberry Pi 4 8GB at 1.5 GHz stock clock, 4 CPU threads, running llama.cpp:
| Model | Params | Prefill tok/s | Generation tok/s | Practical use |
|---|---|---|---|---|
| TinyLlama 1.1B q4_K_M | 1.1B | 10-14 | 4-6 | Fastest option; limited quality |
| Phi-2 2.7B q4_K_M | 2.7B | 6-10 | 2-3.5 | Better instruction following |
| Gemma 2 2B q4_K_M | 2B | 7-11 | 2.5-4 | Good balance |
| Qwen 2.5 1.5B q4_K_M | 1.5B | 8-12 | 3-5 | Best 1B-class for tools |
| Qwen 2.5 3B q4_K_M | 3B | 5-8 | 1.5-2.5 | Ceiling of viable |
| Llama 3.2 3B q4_K_M | 3B | 5-8 | 1.5-2.5 | Ceiling; slow |
| Phi-3 mini 3.8B q4_K_M | 3.8B | 4-6 | 1-2 | Painful |
| 7B any q4_K_M | 7B | 2-4 | 0.4-0.8 | Not viable |
Numbers are broadly consistent with the llama.cpp GitHub issue tracker's ongoing Pi 4 benchmarks. Overclocking to 2.0 GHz with proper cooling adds roughly 15-25% throughput.
What "usable" actually means at 1-3 tok/s
At 2 tok/s a 100-word response takes about 40 seconds. That is unusable for a chat interface where humans expect ChatGPT-tier latency. It is fine for:
- Intent classification for a smart-home assistant ("is this command a light-toggle or a timer?").
- Small summarization tasks where the model is called in the background.
- Slow Discord bot on a private server.
- Home-assistant natural-language date parsing.
- Very small RAG lookups where the retrieval matters more than the generation.
Storage — why the microSD is a mistake
Loading a 1.5-2.5 GB q4 model from a microSD takes 30-60 seconds. Loading it from a USB 3.0 SSD like the Crucial BX500 1TB drops that to 5-10 seconds. The model does not need the SSD once warm — it lives in RAM — but the cold-start time matters if you plan to swap models or reboot often.
A USB 3.0 enclosure with a SATA SSD is the cheapest way to escape microSD hell on the Pi 4. Boot from the SSD too — the Pi 4 has USB-boot support enabled in the current bootloader.
Cooling matters more than you would guess
The Pi 4 throttles hard when the SoC hits 80 C. Under sustained llama.cpp load the CPU stays pinned at 100% across all four cores, which pushes the SoC toward the throttle threshold within minutes in a passive case. A proper active cooler (Argon ONE, Pi 4 Cooling Case with fan) or even a small heatsink plus a 40 mm fan keeps throughput consistent.
Community measurements show throttled Pi 4s losing 20-30% of steady-state throughput vs cooled ones. If you plan to run inference for hours, cool the Pi.
Compared to the RTX 3060 tower and Pi 5
For scale, the same 3B q4 model on a full RTX 3060 12GB tower runs at ~90-120 tok/s generation — roughly 40-60x the Pi 4's throughput. On a Raspberry Pi 5 8GB, the same model hits ~4-6 tok/s at stock clock, 2-3x the Pi 4.
The Pi 4 is not chosen for speed. It is chosen for cost, silence, power budget, and physical footprint. If those constraints matter more than latency, the Pi 4 makes sense. If they do not, buy a real GPU.
Ecosystem — what a Pi 4 LLM node connects to
A Pi 4 LLM node fits well in a home-lab topology alongside:
- Raspberry Pi Zero W sensor nodes handing off natural-language events.
- Home Assistant on the same LAN calling the Pi 4 via a REST endpoint.
- A larger machine on the network — including an AMD Ryzen 7 5800X build with a discrete GPU — that handles the heavy inference while the Pi 4 handles routing.
The point of the Pi 4 in that topology is edge intent classification, not central inference. Route the hard questions to a real machine; keep the easy ones on the Pi.
Practical deployment — the Home Assistant intent-router pattern
The most common real-world use of a Pi 4 LLM node in 2026 is the "small intent-classification model for Home Assistant" pattern. The flow:
- User speaks a natural-language request into a voice-satellite device.
- Whisper (running elsewhere or via Home Assistant Assist) transcribes it.
- The Pi 4 receives the transcript, runs it through a 1.5-3B q4 model with a tight system prompt: "classify this request into an intent and extract parameters."
- The Pi 4 returns JSON:
{"intent":"set_thermostat","room":"office","temp_f":68}. - Home Assistant executes the intent.
At 1.5-3 tok/s generation, a 20-30 token JSON response takes ~10-20 seconds. That is bordering on unusable for voice interaction. The workaround is to shrink the model further — Qwen 2.5 0.5B or TinyLlama 1.1B at q4 hit 4-8 tok/s on a Pi 4, which brings the response time under 3-5 seconds. Quality drops but is acceptable for a narrow intent-classification task.
Batch summarization — the other viable workload
The Pi 4 shines for slow-batch workloads where latency is not a constraint. Examples:
- Nightly summarization of a household's incoming email into a single digest.
- Daily summary of RSS feed items filtered by user interests.
- Weekly rollup of home-camera event logs into human-readable stories.
At 2 tok/s the Pi can produce a 500-word summary in 4-5 minutes. Running that as a nightly cron job costs nothing in perceived latency because nobody is waiting for it.
Model quantization on ARMv8 — what actually works
Not every quantization scheme performs equally on the Pi 4's ARMv8 CPU. llama.cpp's Q4_K_M is the community-recommended sweet spot per multi-year benchmark discussion:
| Quant | Pi 4 8GB tok/s (2.7B Phi-2) | Quality vs fp16 | Notes |
|---|---|---|---|
| Q2_K | ~4 gen | ~70-78% | Fast but shaky |
| Q3_K_M | ~3.5 gen | ~85-88% | Marginal quality |
| Q4_K_M | ~2.8 gen | ~92-94% | Recommended |
| Q5_K_M | ~2.2 gen | ~96% | Best quality that still fits with headroom |
| Q6_K | ~1.7 gen | ~98% | Slow, tight on RAM for 3B+ |
| Q8_0 | ~1.2 gen | ~99% | Not worth the throughput hit |
Q4_K_M is the practical default. Q5_K_M is a modest quality bump if you have RAM headroom.
Overclocking the Pi 4 — worth it?
The community-standard Pi 4 overclock is 2.0 GHz with arm_freq=2000 in /boot/config.txt, paired with a modest 6 mV over_voltage. With adequate cooling this holds stable on most Pi 4 units and delivers 15-25% more llama.cpp throughput. It requires:
- Active cooling (fan or high-mass heatsink).
- A quality PSU delivering the full 3 A at 5.1 V.
- A microSD or SSD with reliable I/O — overclocked Pi 4s are more sensitive to storage timeouts.
For a dedicated LLM-only node, overclocking is worth doing. For a mixed-workload Pi (running Pi-hole, Home Assistant, and inference), stability matters more than 20% extra tok/s.
Power, silence, and always-on economics
At 3 W idle and 7-8 W under load, the Pi 4 costs about $8-10 a year in electricity to run 24/7 at U.S. rates. A silent fanless case makes it invisible on a shelf. That is genuinely hard to beat with any x86 mini PC — even a passive N100 draws ~5-6 W idle and ~15-20 W under load.
For a device that responds to a handful of intent-classification queries a day, the Pi 4 is close to free to run. That is what keeps the platform alive in 2026 despite the Pi 5 and mini-PC alternatives.
Common pitfalls
- Running from microSD. Cold-start latency is painful; storage wear kills the card. Move to USB SSD boot.
- Ignoring cooling. Thermal throttling drops throughput 20-30%. Active cooling is not optional for sustained inference.
- Trying to run 7B. It works but at 0.5 tok/s it is not a real workload. Do not.
- Assuming the Pi's GPU helps. VideoCore VI is not a supported llama.cpp backend. CPU-only.
- Skipping the RAM check. llama.cpp's memory planner assumes at least 6 GB usable; a Pi 4 running Home Assistant + Node-RED + the model server can OOM if you are not careful.
When NOT to use a Pi 4 for LLMs
If your workload needs interactive chat, if your query rate exceeds one every few minutes, or if you want to run a 7B+ model, buy a mini PC. A used Intel N100 mini PC at $150-200 runs 3B models at 10-15 tok/s and 7B at 3-5 tok/s — 5-10x the Pi 4's throughput at 2-3x the price and 3-4x the wall power. That is a better trade for most use cases.
Bottom line
The Pi 4 8GB is a viable LLM node when the workload is edge intent classification, slow batch summarization, or a tightly-scoped side project. It is not viable for interactive chat, agent loops, or anything that needs faster than one paragraph per minute. Do not buy a Pi 4 today to run LLMs — buy a Pi 5, a used mini PC, or a real GPU. But if you already own one and want to give it a low-effort role in a home lab, this workload class fits.
Related guides
- Raspberry Pi 4 8GB Starter Home Lab
- Raspberry Pi 4 Cyberdeck Music Workstation
- Raspberry Pi Zero W Projects
Citations and sources
- Raspberry Pi Foundation — Pi 4 8GB product page and BCM2711 spec
- llama.cpp GitHub — Raspberry Pi benchmark discussion
- r/LocalLLaMA — community measurements on Pi 4 LLM throughput
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
