Can a Raspberry Pi 4 8GB Run a Local LLM? What Actually Works in 2026

Yes, a Raspberry Pi 4 8GB can run a local LLM in 2026 — but only small quantized models in the 1B-3B-parameter range, at a few tokens per second, via llama.cpp or Ollama. It's enough for routing, classification, simple text cleanup, and edge tasks; it's nowhere near a discrete-GPU experience for chat with mid-size models.

What "running an LLM on a Pi" actually means in 2026

The "can I run an LLM on a Pi" question has become perennial in the maker community and gets a frustrating range of answers depending on what the asker means by "run an LLM." If the answer they need is "run something close to ChatGPT on a Pi," the answer is no, and it will continue to be no. If the answer they need is "run a small useful language model on a $80 board for edge work," the answer is yes and has been yes for over a year.

Per the Raspberry Pi Foundation's product page, the Pi 4 8GB ships with a quad-core ARM Cortex-A72 at up to 1.5GHz (overclockable to ~2GHz with good cooling) and 8GB of LPDDR4-3200 system RAM. It has no GPU acceleration for LLM workloads in the consumer-GPU sense — the integrated VideoCore VI is a graphics engine, not an inference accelerator. All inference happens on the CPU, with the LPDDR4 acting as the model storage and KV-cache backing.

That hardware spec defines the practical envelope: models small enough to fit in 8GB of RAM with some headroom for the OS and user processes, and throughputs constrained by ARM CPU compute and DDR4 bandwidth (~26 GB/s theoretical, less in practice).

Key takeaways

• The Pi 4 8GB realistically runs models in the 1B-3B parameter range at q4-q5 quants.
• Token throughput is in the few-tokens-per-second range for small models, slower for larger ones.
• Active cooling (heatsink + fan or cooler case) is required for sustained inference.
• A Pi is useful for edge tasks, classification, intent routing, sensor responses, and learning.
• For real LLM work, a discrete card like the RTX 3060 12GB is far more capable.
• Perf-per-watt is the Pi's real advantage: ~7W sustained vs ~170W for a discrete GPU.
• llama.cpp is the canonical runner; Ollama layers on top.

What you'll need to run an LLM on a Pi 4

A cheap SD card with weak sustained-write speeds will bottleneck model loading; a USB 3.0 SSD makes the experience noticeably better.

Which model sizes fit in 8GB?

Quantized models in the GGUF format are the right target. Approximate sizes after q4 quantization:

In an 8GB Pi 4 with ~6.5GB usable after OS overhead, 1B-3B models leave comfortable headroom for KV-cache and short contexts. 7-8B models technically fit but leave little room for context and run at the lower end of acceptable throughput.

How many tokens per second can a Pi 4 produce?

Community measurements published on the llama.cpp GitHub repository issues and Reddit r/LocalLLaMA put a Pi 4 8GB running a 1-3B q4 model in the rough range of 3-8 tokens per second on the CPU, with the higher end requiring optimized builds, NEON SIMD, and active cooling to prevent throttling. A 7B q4 model on the same hardware drops to roughly 1-3 tokens per second, which is below conversational fluency.

For comparison: an RTX 3060 12GB on the same model runs at 15-40 tokens per second for 7B models and 40-80 tokens per second for 1-3B models — an order of magnitude faster, and that's before considering the larger models the 12GB card can hold.

The Pi is not competitive with a discrete GPU for raw throughput. What it is competitive on is power: a Pi running sustained inference draws ~6-8W; the discrete GPU draws ~170W. For 24/7 background tasks, the Pi's perf-per-watt is genuinely useful.

Pi 4 8GB CPU inference versus RTX 3060 12GB GPU inference

Same direction-of-magnitude pattern: the Pi is a learning, edge, and orchestration board; the discrete GPU is a serious local-LLM platform. The 8GB Pi 4 model number is unchanged but the ecosystem and quantized models have improved enough to make small-model inference practical where it wasn't a year ago.

Quantization on the Pi

q4_K_M is the workhorse quant for Pi-class inference. It keeps the model small enough to leave KV-cache headroom and runs at acceptable speeds on ARM CPUs. q5 is slightly larger and slower but closer to fp16 quality; q3 saves more memory at noticeable quality cost.

Practical guidance:

• 1-3B models: use q4_K_M or q5_K_M. Both fit comfortably.
• 7B models: use q4_K_M only; q5 is too tight.
• 8B+ models: use q4 or q3, expect slow throughput.

The Ollama catalog ships pre-quantized variants of major open models; pulling them is a one-line command. llama.cpp users can convert their own GGUFs from upstream weights, but for most users Ollama's pre-built models are easier and equivalent.

When to offload to a desktop GPU instead

The cardinal rule: don't run a model on the Pi that fits comfortably on the discrete GPU you also have. The Pi makes sense for:

1. Tiny models running 24/7. Where the desktop being on isn't desirable.
2. Distributed tasks. Routing requests to a larger upstream model.
3. Edge applications. Sensors, IoT, voice assistants where power and latency to a remote box matter.
4. Learning. Cheap board for experimenting with the local-LLM stack.

If you have an RTX 3060 12GB or ZOTAC 3060 12GB box and the question is "where should I run a 7B chat model," the answer is the discrete GPU, every time. The Pi's role is supplementary, not primary.

What real homelab jobs is Pi-class inference good for?

• Intent classification. "Is this email a support ticket or marketing?" — 1B model nails it.
• Text cleanup. "Strip HTML and summarize to 100 chars" — easy at Pi scale.
• Sensor-triggered responses. Door camera says "person detected"; Pi generates a notification text.
• Routing. Small model decides whether to dispatch a query to a larger upstream model.
• Offline voice assistants. Wake-word detection + small-model dialog management.
• Always-on background tasks. Tagging, filing, indexing — slow but constant.

What it's not for: interactive chat with mid-size models, multi-turn agentic work, document QA over large contexts, anything where latency-to-first-token matters and the model is bigger than 3B.

Perf-per-watt: the Pi's real advantage

For a 24/7 always-on tiny-model job, the Pi runs at ~$8-15/year in electricity. The discrete GPU running at idle most of the time and spinning up only when needed sits in a different operational pattern — typically more cost-effective per token at any meaningful query volume, but more expensive at idle.

The Pi wins specifically when (a) the work is small and constant, and (b) you don't already have a desktop running anyway.

Cooling: not optional

Sustained inference pins the Pi 4's CPU at 100% and generates real heat. Without active cooling, the SoC hits its thermal limit (typically 80°C) and the Pi throttles, sometimes within 60 seconds. A heatsink + small fan case or an aluminum heatsink case is essential for any sustained LLM workload.

Per maker community measurements, a passively-cooled Pi 4 hits throttling in under a minute of llama.cpp inference; the same Pi with a fanned case sustains full clocks indefinitely. The fan and case add $10-15 to the build — well worth it.

Worked example: a routing Pi feeding a desktop GPU

A pattern many homelabbers use that takes full advantage of the Pi's strengths:

1. A Pi 4 8GB runs always-on, hosting a small 1-3B model with Ollama.
2. Incoming requests (voice, webhook, automation) hit the Pi first.
3. The Pi's small model classifies intent and decides: handle locally, or escalate to the desktop's larger model.
4. Routed requests go to the desktop with the 12GB RTX 3060.
5. The desktop sleeps most of the time, waking only when escalated.

This pattern combines the Pi's perf-per-watt advantage (always on at 7W) with the desktop's throughput advantage (powerful only when needed). It's a real architecture, not a toy, and it scales naturally to multiple small Pi nodes feeding a single beefy inference server.

Common pitfalls

• Underpowered PSU. Cheap 5V/2.5A PSUs cause clock drops under load. Use the official 5V/3A USB-C PSU.
• Cheap SD card. Class 10 is the minimum; a class A2 microSD or a USB 3.0 SSD massively improves model load times.
• Wrong llama.cpp build. ARM-NEON-optimized builds are 2-3x faster than generic builds. Make sure your binary was compiled with NEON enabled.
• Too-big model. Loading a 7B model in an 8GB Pi leaves only ~1GB for KV-cache; long contexts OOM.
• Default CPU governor. Some distros default to power-saving; switch to "performance" governor for sustained inference.

When NOT to bother with Pi LLMs

Skip the Pi LLM path if:

• You need conversational chat with mid-size models.
• You already own a desktop with a GPU and the Pi would just duplicate the work.
• Your task is latency-sensitive.
• You're not interested in tinkering with quantization and runner builds.

For real chat work, save for a used RTX 3060 12GB or ZOTAC 3060 12GB. For learning the stack on a board you can dedicate, the Pi is great.

Power, storage, and a few small build details

A practical Pi 4 8GB LLM rig in 2026 totals around $130-160 once accessories are factored in: the Pi 4 8GB board itself, an active-cooled case ($15-25), a 5V/3A official USB-C PSU ($10-15), a USB 3.0 SSD or fast microSD ($25-50), and a short Ethernet cable. That's the cheapest viable always-on LLM appliance, drawing about 7-9 watts under load.

Bottom line

A Raspberry Pi 4 8GB is a credible platform for small-model local LLM work in 2026. It runs 1-3B parameter models at q4 quants at usable throughput, handles always-on background tasks at low power, and is a great learning platform for the local-LLM stack via llama.cpp and Ollama.

It is not a replacement for a discrete GPU on real chat or agentic work. Pair it with active cooling, the official PSU, and a fast storage option, and treat it as a complementary edge node — not your primary local inference platform.

Related guides

• GLM-5.2 local GPU VRAM requirements
• Ollama vs llama.cpp on a 12GB GPU
• Which GPU runs which LLM

Citations and sources

• Raspberry Pi Foundation — Pi 4 Model B product page
• llama.cpp GitHub repository — runner documentation and ARM optimizations
• Ollama — model catalog and runner documentation

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.