Build a Home AI Assistant on a Raspberry Pi 4 8GB in 2026

Yes, a Raspberry Pi 4 8GB can run a local AI assistant in 2026 using 1-3B parameter models at q4 quantization, achieving 3-8 tokens/second on-device — enough for short-answer voice or text assistants. For anything heavier (7B+ or long-context agents), the Pi does the wake-word and routing while a desktop rig with an RTX 3060 12 GB handles the actual inference.

What a Pi-class assistant can and can't do

The 2020s expectation of "AI assistant" — set by cloud services like ChatGPT and Google Assistant — is set embarrassingly high compared to what a Raspberry Pi can locally deliver. A cloud service runs a 70B+ frontier model on GPUs that draw 300+ watts. The Pi 4 8GB has a 1.5 GHz quad-core Cortex-A72 that draws about 6 watts under load. There is no version of this hardware that runs a frontier model.

What the Pi can do is genuinely useful in the right shape. Small language models — TinyLlama 1B, Phi-2 2.7B, Phi-3-mini 3.8B, Qwen2.5-1.5B, Llama-3.2 1B — run entirely locally on the Pi's 8 GB of RAM at q4 quantization. They answer short questions competently, respond to voice queries after wake-word activation, and handle routing decisions ("should I forward this to my desktop?") without a network hop.

Combine that with a home-lab desktop running a bigger model behind Ollama or LM Studio, and you get a two-tier assistant: the Pi handles wake-word and always-on presence at 6 watts; the desktop handles heavy lifting only when needed, spinning up its GPU on demand.

We wrote this for makers and home-lab enthusiasts building an assistant they actually control end-to-end — data stays local, no cloud API bills, no vendor lock-in. It's not a Siri clone; it's a small custom brain you built yourself.

What you'll need — checklist

• Raspberry Pi 4 Model B, 8GB RAM — 4 GB works for smaller models but 8 GB gives you room to breathe with 3B-class models.
• Fast storage. The Pi's stock microSD works but is a bottleneck; a SanDisk Ultra 3D 1 TB SSD paired with a FIDECO SATA-to-USB 3.0 adapter gives you 400+ MB/s effective read for model files and system storage.
• 5V/3A USB-C power supply (official Raspberry Pi supply recommended).
• Active or passive cooling. Under sustained inference the SoC will thermal-throttle; a 3D-printed case with a low-noise 40 mm fan solves this.
• USB microphone (any decent USB mic works; even $20 desktop mics are fine for voice interface).
• Speaker or 3.5 mm headphone output.
• Wired Ethernet strongly preferred over Wi-Fi for reliability.

Key takeaways

• 1-3B models at q4 run locally at 3-8 tok/s on a Pi 4 8GB. Enough for short-answer assistants.
• 7B+ models are impractical to run locally on the Pi — memory pressure and per-token latency both hit walls.
• Two-tier architecture is the winning pattern: Pi handles wake-word / routing, a desktop rig handles heavy inference on demand.
• Storage is a real bottleneck: skip the microSD, use a SATA SSD via USB 3.0.
• Perf-per-watt is unmatched for always-on presence: the Pi at 6 watts costs $6-7 per year of electricity.

Which small models actually run on a Pi 4 8GB?

Approximate tokens-per-second measured with llama.cpp on Raspberry Pi OS 64-bit, with the Pi lightly overclocked to 1.8 GHz and thermally stable:

The llama.cpp project's ARM-optimized kernels make all of this viable; without them, Pi inference would be an order of magnitude slower. Phoronix has a rolling series of ARM/Pi benchmarks at phoronix.com that track llama.cpp performance on the Pi 4 and Pi 5.

For interactive voice assistants, 5+ tok/s is the practical minimum — under that, response latency (10-15 second reply to a short query) breaks the "assistant" feel. That puts you on the 1-2B tier for a fully local Pi assistant. For longer-form questions where a 20-second reply is acceptable, the 3B tier opens up.

Pi 4 8GB compute + memory budget

Memory bandwidth is the primary constraint. LLM inference at q4 is memory-bandwidth-bound; the Pi's ~14 GB/s ceiling is roughly 25x slower than a desktop RTX 3060 (360 GB/s). That ratio maps directly to tok/s: a 3060 hitting 40 tok/s on a 7B model → the Pi at 1.5 tok/s on the same, if it fits.

The 8 GB variant is the only Pi 4 SKU worth buying for AI work. 4 GB fits smaller models but you'll spend more time swapping and less time inferencing. 8 GB gives you room for a 3.8B model plus OS plus your app.

Storage matters: pairing an SSD via SATA-to-USB

Loading a 1-4 GB model file from a microSD takes 30-60 seconds. Loading from a USB 3.0-connected SATA SSD takes 3-6 seconds. That difference matters when you're iterating on which model to use, and it matters more when your assistant app auto-restarts and needs to warm up quickly.

Pair a SanDisk Ultra 3D 1 TB SSD with a FIDECO SATA-to-USB 3.0 adapter for ~$205 combined. That gives you enough space for the OS, half a dozen model files, and your app code. Speed lands around 400 MB/s effective — enough that model load times feel snappy, and if you enable ZRAM swap (you should), you get an escape valve for models that briefly exceed the 8 GB RAM.

Alternatively, some builders boot the entire Pi OS from the SSD (Pi 4 supports USB boot with a BIOS update) which lets you skip the microSD entirely and get faster overall system I/O.

Benchmark table: Pi 4 vs offloading to a desktop

The reference "big brother" desktop is a ZOTAC RTX 3060 12GB paired with a 5800X or 5700X, running Ollama.

The Pi handles wake-word at parity with anything — that's not a compute-bound task. The Pi wins on always-on power draw. The desktop wins on any non-trivial inference workload.

When to keep it local vs route to the desktop

Rule of thumb: short queries stay on the Pi; long-form or reasoning-heavy queries route to the desktop.

Split points:

• "What time is it?" / "Turn on the kitchen light." → Pi handles it, no round-trip.
• "Summarize the top three headlines this morning." → route to desktop for the model that can produce useful summaries.
• "Write a birthday message for my sister." → route to desktop; needs enough model quality to be worth reading.
• "Set a timer for 20 minutes." → Pi handles it locally.
• "What's a good recipe for the salmon in my fridge?" → route to desktop.

The routing logic itself can be a Pi-hosted small model. TinyLlama at 1B is competent at classifying incoming queries as "trivial → answer locally" vs "complex → forward." That's the two-tier architecture in one paragraph.

Wiring it into a voice / text front end

For voice, the standard open-source stack in 2026 is:

• Wake-word detection: Porcupine (free tier) or openWakeWord (fully open-source). Both run comfortably on the Pi.
• Speech-to-text: whisper.cpp with a tiny.en or base.en model runs at real-time speed on the Pi.
• Small LLM inference: llama.cpp with your chosen 1-3B model.
• Text-to-speech: Piper (open-source, runs on-device) produces broadcast-adjacent speech in real time on the Pi.
• Home-automation integration: hook into Home Assistant if you're already running it. Otherwise, MQTT or HTTP directly to your devices.

For text-only front ends (Slack bot, Discord bot, personal chat), skip the STT/TTS stack and connect the LLM's OpenAI-compatible API directly to the chat client's webhook.

Perf-per-watt: why the Pi makes sense for always-on

A Pi 4 8GB idles at 3 W and peaks around 8 W during inference. Over a year of 24/7 operation at ~5 W average, that's about 44 kWh — $6-7 in electricity at typical US rates.

A desktop rig with an RTX 3060 12 GB idling at 60 W with the model resident (kept warm for low latency) is ~525 kWh/year — $70-90. Adding wake-suspend logic can cut this dramatically, but at the cost of first-reply latency (models take 3-6 seconds to load from cold).

For pure always-on wake-word + routing, the Pi is the correct hardware. For actual heavy inference, spin up the desktop only when needed and let it sleep between requests.

Common pitfalls

• Thermal throttling. The Pi 4's SoC will throttle if it reaches 82°C. Under sustained inference, a passive case will throttle within 30 seconds. A small active fan or a proper heatsink fixes this.
• microSD as primary storage. Model loads take 10x longer, and microSD wear-out under LLM read cycles becomes a real problem in 12+ months. SSD is the correct choice.
• Assuming the Pi can run any model on Ollama. Ollama on ARM works but is less polished than on x86. llama.cpp compiled from source with NEON optimizations is often the more reliable path on the Pi.
• Forgetting to enable ZRAM swap. Some models briefly spike past 8 GB during load. ZRAM swap catches the spike; without it you get OOM kills.
• Wi-Fi during heavy inference. The Pi's Wi-Fi and internal bus contend for power under load; use wired Ethernet for a home assistant.
• Ignoring wake-word compute. Wake-word runs 24/7 at 3-5% CPU; running it alongside a 3B model at inference time can slow both. Pin wake-word to one core and the LLM to the others.

Bottom line: realistic BOM

A sensible 2026 BOM for a Pi-hosted always-on assistant:

• Raspberry Pi 4 8GB: ~$85
• SanDisk Ultra 3D 1 TB SSD: ~$85
• FIDECO SATA-to-USB 3.0 adapter: ~$24
• Official Pi 4 USB-C 5V/3A power supply: ~$8
• Active-cooled Pi 4 case (Argon ONE, Flirc, similar): ~$20
• USB microphone: ~$25
• 3.5 mm speaker (or Bluetooth speaker): ~$25

Total: ~$270 for the full BOM. Compare that to any commercial cloud-connected assistant: comparable one-time hardware cost, zero recurring cloud fees, complete data locality.

If you need the heavy-lifting tier too — for those "write me a birthday message" queries — spin up a desktop with the RTX 3060 12 GB as a second-tier compute node on the same LAN. The Pi does the wake-word and routing; the desktop wakes up on demand and returns the answer within 2-3 seconds.

Related guides

• Which GPU Runs Which LLM? — the desktop side of a two-tier setup
• Ollama vs LM Studio on a 12 GB RTX 3060 — pick a runner to expose an API for the Pi to call
• GLM-5.2 vs Frontier Models — model choice for the desktop tier
• Best Controller for Emulation and Fighting Games — a Pi is also great for retro gaming

Citations and sources

• Raspberry Pi 4 Model B product page
• ggml-org/llama.cpp on GitHub
• Phoronix — ARM / Raspberry Pi benchmarks

_As of 2026. Small-model quality and tok/s numbers improve every few months as new architectures land; the Pi hardware itself is stable._