Building a Local LLM Voice Assistant on Raspberry Pi 4 8GB: Whisper + Llama 3.2 1B Setup (2026)

You can build an offline LLM voice assistant on a Raspberry Pi 4 8GB using Whisper-tiny for speech-to-text and Llama 3.2 1B at q4_K_M quantization for inference. Total round-trip latency from end-of-speech to spoken response is 4–7 seconds in a quiet room. It works for kitchen-timer-tier commands and simple factual queries; it does not work for freeform conversation or multi-turn reasoning. The Raspberry Pi 4 8GB (B0899VXM8F) is the minimum RAM spec — the 4GB model does not fit the full Whisper+LLM+TTS pipeline simultaneously.

Why Local STT + LLM Beats Alexa for Tinkerers

Alexa, Google Assistant, and Siri send your audio to remote servers. For projects in sensitive environments — workshop conversations, medical reminders, offline deployments — that's a non-starter. Local inference on a Pi is genuinely private: nothing leaves the device.

The tradeoff is latency and capability. A Pi 4 at q4_K_M produces ~2.5 tokens/sec vs 50+ tokens/sec on an M3 MacBook or RTX 3070 GPU. But for a voice assistant that responds to "Set a timer for 10 minutes," "What is the capital of Thailand," or "Turn on the lights" — 2.5 tok/s is sufficient if you're willing to wait 5–7 seconds.

Key Takeaways

• Pi 4 8GB ceiling: q4_K_M of Llama 3.2 1B (GGUF, ~700MB RAM) is the largest practical model
• Whisper-tiny: real-time factor 1.4x on Pi 4 (a 5-second audio clip takes 7 seconds to transcribe)
• Llama 3.2 1B q4_K_M: ~2.5 tok/s on Pi 4 8GB; native context is 128K tokens but cap at -c 4096 or -c 8192 to keep the KV cache in RAM
• Piper TTS: ~0.4s synthesis for a 10-word sentence, offline, no cloud dependency
• Round-trip latency: 4–7s total (STT 1–2s + LLM prefill 1s + LLM generation 1–3s + TTS 0.4s)

Hardware Bill of Materials

Total hardware cost: ~$160–200 for a full build. A budget USB microphone at $20 works fine — the HyperX is overkill unless you also use it for recording.

Why the Raspberry Pi 4 Needs 8GB for Local LLM Work

Memory budget for the voice pipeline at idle:

• Raspberry Pi OS (64-bit, lite): 300–400MB
• Whisper-tiny GGUF model: 75MB
• whisper.cpp process overhead: 100MB
• Llama 3.2 1B q4_K_M GGUF: 700MB
• llama.cpp process overhead: 200MB
• Piper TTS model (en_US-lessac-medium): 60MB
• Python glue script + buffers: 150MB
• OS headroom: 300MB

Total: ~1,885MB, with a comfortable margin under the 8GB limit. On a 4GB Pi, after OS and Whisper, you have ~2.8GB remaining — not enough for Llama 3.2 1B q4_K_M plus TTS simultaneously. You'd be forced to run Llama 3.2 1B at q2_K (~430MB) with significant quality degradation, or skip TTS and output text only.

Quantization Matrix: Llama 3.2 1B on Pi 4 8GB

Recommendation: q4_K_M is the practical sweet spot — best quality within the RAM budget at the highest usable token rate for the pipeline.

Whisper Model Sizing: Tiny vs Base vs Small

Real-time factor means "time to transcribe" / "audio length." At 1.4x, a 3-second voice command takes 4.2 seconds to transcribe — acceptable. At 9.1x, a 3-second command takes 27 seconds — unusable for voice UX. Use whisper-tiny on Pi 4; whisper-base is acceptable only if your pipeline tolerates 5–6 second STT latency.

WER of 8.1% means about 8 out of 100 words are misrecognized in controlled conditions. In a noisy workshop, expect 12–18% WER with whisper-tiny. Use a directional cardioid mic and speak clearly for best results.

Step-by-Step: Install llama.cpp + whisper.cpp + Python Glue

1. Install prerequisites

sudo apt update && sudo apt install -y build-essential cmake git python3-pip portaudio19-dev
pip3 install pyaudio sounddevice numpy

2. Build llama.cpp (ARM-optimized)

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS
cmake --build build --config Release -j4

Build time: ~8 minutes on Pi 4 8GB. The -DLLAMA_BLAS=ON flag enables BLAS matrix operations that improve throughput ~15–20% on ARM vs the default scalar path.

3. Download Llama 3.2 1B GGUF

wget https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf

4. Build whisper.cpp

git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
make -j4
bash models/download-ggml-model.sh tiny

5. Install Piper TTS

pip3 install piper-tts
python3 -m piper.download_voices --language en_US --quality medium

6. Python glue script (minimal)

import subprocess, sounddevice as sd, numpy as np, wave, tempfile, os

def record_until_silence(seconds=5):
 audio = sd.rec(int(seconds * 16000), samplerate=16000, channels=1, dtype='int16')
 sd.wait()
 return audio

def transcribe(audio_path):
 result = subprocess.run(
 ["whisper.cpp/main", "-m", "whisper.cpp/models/ggml-tiny.bin",
 "-f", audio_path, "--output-txt", "--no-prints"],
 capture_output=True, text=True)
 txt_path = audio_path + ".txt"
 return open(txt_path).read().strip() if os.path.exists(txt_path) else ""

def query_llm(prompt):
 result = subprocess.run(
 ["llama.cpp/build/bin/llama-cli", "-m", "Llama-3.2-1B-Instruct-Q4_K_M.gguf",
 "-p", f"<|user|>{prompt}<|assistant|>", "-n", "80", "--temp", "0.1"],
 capture_output=True, text=True)
 return result.stdout.split("<|assistant|>")[-1].strip()

def speak(text):
 subprocess.run(["piper", "--model", "en_US-lessac-medium.onnx",
 "--output_raw", "-"], input=text.encode(),
 stdout=subprocess.PIPE)

This is a minimal functional example — add VAD (voice activity detection) for production use to avoid recording silence.

Latency Budget Breakdown

The dominant latency is LLM generation — even at 2.5 tok/s, a 60-token response takes 24 seconds. Keep responses short: system-prompt your model with "Answer in 20 words or fewer. Be direct." This cuts generation latency to 8 seconds for typical command responses.

Power Draw and Thermals

A heatsink is not optional. Without active cooling, the Pi 4 progressively throttles from 1500MHz toward 1000MHz starting at 80°C, dropping further toward 600MHz as it approaches the 85°C hard limit — enough to substantially increase LLM inference time and break the latency budget. The official Pi Active Cooler (raspberry pi active cooler) brings peak temps to 71°C under full load. The FREENOVE kit includes a basic heatsink; add a small 5V fan for continuous pipeline use.

How Does This Compare to Pi 5 + Hailo-8L AI HAT?

The Raspberry Pi 5 with the Hailo-8 AI HAT+ (26 TOPS NPU) achieves roughly 8–12 tok/s on Llama 3.2 1B — about 4x faster than the Pi 4. Whisper-tiny transcription drops to real-time factor 0.3x. The full voice pipeline round-trip drops to 1.5–3 seconds, which is a qualitatively different UX. If your project warrants the investment ($80 Pi 5 + $70 Hailo HAT+ = $150 additional), the Pi 5 + Hailo is the correct upgrade path.

Bottom Line

The Pi 4 8GB voice assistant works for kitchen-timer-tier commands and simple factual queries at 4–7 second latency. It does not work for freeform conversation. Use Whisper-tiny + Llama 3.2 1B q4_K_M + Piper TTS as described above. Key risks: throttling without active cooling, WER degradation in noisy environments, and RAM-driven context caps (running at -c 8192 to keep KV cache in budget means long conversations overflow after 30–40 exchanges, even though the model itself supports 128K).

Sources: llama.cpp GitHub repository, whisper.cpp benchmarks, r/LocalLLaMA Pi-4 benchmark threads.

Related Guides

• Best USB Microphone for Maker Projects

Products mentioned

• Freenove Ultimate Starter Kit for Raspberry Pi 5 4 B 3 B+…
• Freenove Ultimate Starter Kit for Raspberry Pi 5 4 B 3 B+…