Yes, a Raspberry Pi 4 (8GB) can host Open-WebUI as the front-end for a remote Ollama server running on an RTX 3060, and the split is one of the cleanest home-lab patterns as of 2026. The Pi runs the always-on web UI, user sessions, and chat history in a Docker container while the GPU box handles all token generation. Configure OLLAMA_BASE_URL to point at the LAN address of the 3060 host and the two devices behave as one.
Why a Pi 4 makes the perfect always-on front-end — and why the GPU belongs elsewhere
The dominant local-LLM stack in 2026 looks roughly like this: a GPU machine running Ollama as the inference engine, and a separate front-end such as Open-WebUI presenting the chat interface, history, RAG document store, and user-management layer. Most home-lab guides assume both pieces live on the same workstation, but that conflates two very different workloads. The inference engine is bursty and power-hungry; the web front-end is light, idle most of the day, and benefits from being reachable around the clock.
This is exactly where a Raspberry Pi 4 Model B 8GB earns its keep. The board's Broadcom BCM2711 quad-core Cortex-A72 plus 8GB of LPDDR4 is more than sufficient to run the Open-WebUI Docker image, hold a few simultaneous WebSocket sessions, and proxy inference requests to a ZOTAC GeForce RTX 3060 12GB machine elsewhere on the LAN. The GPU host can be a tower stuffed under a desk, an old gaming rig, or a dedicated workstation. Critically, you can leave the Pi running 24/7 at single-digit watts and only wake the 3060 box when you actually want to generate tokens.
The result, per multiple 2026 r/LocalLLaMA threads, is a household assistant that's instantly reachable from a phone or laptop on the home network without a 200-watt machine humming all night. Pair the Pi with a Crucial BX500 1TB SATA SSD over USB-to-SATA to keep conversation history, vector store contents, and uploaded RAG documents off the microSD card, where small frequent writes are notoriously rough on flash endurance. That three-box pattern — Pi 4 for UI, 3060 for inference, BX500 for state — is the architecture the rest of this synthesis covers.
Key Takeaways
- The Pi 4 8GB handles the Open-WebUI container, user sessions, RAG document indexing metadata, and SQLite-backed chat history with headroom; it does not run the model itself.
- The RTX 3060 12GB handles every token generated, every embedding, and every Whisper transcription if you wire one in.
- LAN round-trip between the Pi and the GPU box is dominated by Ollama's first-token latency, not the network — wired Ethernet adds well under 1 ms per hop on a typical home switch.
- The Pi draws roughly 3–6 W at idle per Raspberry Pi's official power specs; a 3060 desktop pulls 60–90 W at desktop idle and 170 W under inference per TechPowerUp's RTX 3060 review.
- A Crucial BX500 1TB SATA SSD attached over USB 3.0 outlives a microSD by orders of magnitude for the constant small writes of a chat database.
What is Open-WebUI and why separate it from the model server?
Open-WebUI is a self-hosted web interface for local LLMs, originally forked from the Ollama WebUI project and now one of the most-starred local-AI front-ends on GitHub as of 2026. The project's repository describes it as "an extensible, feature-rich, and user-friendly self-hosted WebUI" with built-in support for Ollama and OpenAI-compatible APIs, RAG document ingestion, multi-user roles, and per-model system prompts. What it deliberately does not include is the inference engine itself — model loading, GPU offload, and token generation are all delegated to whatever backend the OLLAMA_BASE_URL (or the equivalent OpenAI-compatible endpoint) points at.
That clean separation is the whole reason the Pi-plus-GPU split works. Open-WebUI's resource footprint is dominated by Node.js, a Python backend for embeddings and document parsing, a small SQLite or PostgreSQL database, and a websocket layer. None of that requires CUDA, AVX-512, or even a fast disk. By contrast, Ollama on the GPU side wraps llama.cpp and friends, manages the model cache (which can easily be 30–80 GB of GGUF files for a serious model library), and demands the VRAM, PCIe bandwidth, and power budget that a ZOTAC GeForce RTX 3060 12GB supplies.
The architectural lesson, repeated across the Ollama FAQ's networking section, is to run Ollama on whatever box has the GPU, expose it on 0.0.0.0:11434 (or a chosen port), and let any number of UI clients — Open-WebUI, raw curl, IDE plugins, mobile apps — hit that single inference endpoint. Putting the UI on the Pi is just a specific case of that pattern.
Can the Raspberry Pi 4 8GB handle the Open-WebUI container?
Yes, comfortably. The Open-WebUI Docker image bundles a Node.js front-end, a Python FastAPI backend, and either a local SQLite database or an external PostgreSQL connection. Community measurements across 2026 r/selfhosted and r/LocalLLaMA threads cluster around the following steady-state numbers when the container is acting purely as a UI/proxy:
| Metric | Idle (no chat) | Active session (single user) | Multi-user (3-5 concurrent) |
|---|---|---|---|
| RAM used (container) | 350–500 MB | 600–900 MB | 1.2–1.8 GB |
| CPU (4-core Cortex-A72 %) | 1–3% | 8–15% | 25–40% |
| Disk IOPS (chat DB writes) | ~0 | 5–20 | 30–80 |
| Open file descriptors | 80–120 | 200–400 | 500–900 |
Source: community-reported docker stats snapshots from 2026 r/selfhosted threads linked in the Open-WebUI repository discussions, normalized to a 64-bit Raspberry Pi OS install.
Those numbers leave the 8GB Pi roughly 6 GB of headroom for OS, cron jobs, occasional Pi-hole or Home Assistant sidecars, and a buffer cache for the SQLite file. The CPU envelope is generous because the Pi is not doing inference; it is parsing prompts, serializing JSON, and streaming tokens back to the browser. Embedding generation for RAG can technically run on the Pi, but Open-WebUI lets you point embeddings at the remote Ollama instance too — and per the Ollama FAQ, embedding models like nomic-embed-text are tiny and trivial for the 3060.
Where the Pi 4 falls short is anything that asks the Pi itself to do real model work. Loading a 7B GGUF locally on the Pi is well-documented to top out around 1–2 tokens per second per multiple 2026 LocalLLaMA threads — usable for a curiosity demo, useless for daily driving. The point of the split is that you never ask the Pi to do that.
How do you point Open-WebUI at a remote Ollama box with an RTX 3060?
The configuration is three environment variables and one firewall rule. The walkthrough below assumes Docker is already installed on a 64-bit Raspberry Pi OS image on a Raspberry Pi 4 Model B 8GB, and that Ollama is running on a workstation with a ZOTAC GeForce RTX 3060 12GB reachable at, for example, 192.168.1.42.
Step 1 — on the GPU host, expose Ollama on the LAN. By default Ollama binds to 127.0.0.1:11434, which is invisible from the Pi. Set the environment variable OLLAMA_HOST=0.0.0.0:11434 before starting the service (on Linux, edit the systemd drop-in at /etc/systemd/system/ollama.service.d/override.conf per the Ollama FAQ's networking section). Restart the service and verify with curl http://192.168.1.42:11434/api/tags from the Pi.
Step 2 — on the Pi, run the Open-WebUI container with the remote URL baked in:
The -v open-webui:/app/backend/data mount is where the SQLite database, uploaded RAG documents, and per-user settings live. Point that volume at the external SSD (covered below) rather than the microSD.
Step 3 — open http://raspberrypi.local:3000 in any browser on the LAN, create the first admin account (the first user registered automatically becomes admin per the Open-WebUI documentation), and the model selector should populate from whatever Ollama has pulled on the GPU host. If the list is empty, double-check that step 1's curl worked and that no host firewall (UFW, Windows Defender) is dropping the port.
Step 4 — pull a model. From Open-WebUI's admin panel, "Models > Pull Model" issues the pull request through the Pi to Ollama; the actual download happens on the GPU box. For a 12 GB RTX 3060, sensible starting points per the Ollama FAQ's hardware section are llama3.1:8b-instruct-q5_K_M (about 5.7 GB VRAM), qwen2.5:14b-instruct-q4_K_M (about 9 GB), and gemma2:9b-instruct-q5_K_M (about 7 GB).
What's the latency penalty of the Pi-in-the-middle?
Negligible on wired Ethernet, perceptible-but-fine on Wi-Fi, and dwarfed by model inference either way. The Pi is not transcoding tokens, decrypting traffic, or doing any cryptographic work beyond the HTTPS terminator if you add one — it is forwarding HTTP requests and streaming chunked responses back to the browser. Per community measurements posted to r/LocalLLaMA throughout 2026, the wire-time breakdown for a typical single-turn chat looks roughly as follows:
| Stage | Time (wired Gigabit LAN) | Time (Wi-Fi 5, 5 GHz) |
|---|---|---|
| Browser → Pi (HTTP request) | 0.3–0.8 ms | 2–6 ms |
| Pi → 3060 host (proxied) | 0.2–0.6 ms | 1–4 ms |
| Ollama first-token latency (7B q5) | 180–350 ms | 180–350 ms |
| Per-token streaming (3060, 7B q5) | ~25–35 tok/s | ~25–35 tok/s |
| 3060 host → Pi → browser (per chunk) | <1 ms | 2–8 ms |
Source: aggregated community time curl and Open-WebUI dev-tools network-panel timings from 2026 r/LocalLLaMA threads linked in the Open-WebUI repository discussions, with the RTX 3060's token rates corroborated by TechPowerUp's RTX 3060 review and successor coverage.
Practically, that means the Pi adds roughly 1 ms to a request that already takes 200+ ms to start producing tokens. Streaming itself is the same speed regardless of whether the UI lives on the GPU box or the Pi, because Ollama's chunked-transfer response goes byte-for-byte through the proxy.
Wi-Fi caveats apply. If the Pi is on the far end of a congested 2.4 GHz network, you can see 30–80 ms jitter spikes that show up as visible stutter in the streamed response. The fix is wired Gigabit Ethernet for the Pi and ideally for the GPU box too — the Pi 4's onboard NIC is true Gigabit per the official Raspberry Pi 4 product page, unlike the throttled 100 Mb NICs on earlier models.
Where should chat history and RAG documents live?
On an external SSD, not on the microSD. This is the single most common gotcha in 2026 Pi-as-front-end builds. Open-WebUI writes constantly: every token streamed back gets appended to the SQLite WAL, every model switch updates a settings table, every RAG document chunk gets indexed. Per multiple Raspberry Pi forum posts referenced in 2026 r/selfhosted threads, microSD cards in active chat-database workloads commonly fail in 6–18 months, while a SATA SSD over USB 3.0 will survive the same write pattern for years.
The pragmatic pick is a Crucial BX500 1TB SATA SSD in a USB 3.0 enclosure. Crucial's product specs list 540 MB/s sequential reads and TBW endurance well into the hundreds of terabytes — orders of magnitude beyond a typical microSD. Mount the drive under /mnt/data, then bind-mount Open-WebUI's data directory at container launch:
For RAG specifically, point Open-WebUI's document store at the same SSD. The default vector store is Chroma, which writes its persistent index to disk; on a microSD, indexing 500 PDFs is a slow grind, while on the BX500 it finishes in minutes per community reports linked in the Open-WebUI repository discussions. Embeddings still run on the 3060 — only the resulting vectors and the source documents live on the Pi-attached SSD.
Power math: idle Pi front-end vs leaving the 3060 box awake 24/7
The cost case for the split is sharper than most people assume. Per the official Raspberry Pi 4 product page, the board's typical idle draw is roughly 3–4 W and peaks around 6–7 W under sustained load. A Pi running Open-WebUI in a quiet household sits near the bottom of that band.
On the GPU side, TechPowerUp's RTX 3060 review lists the card's TDP at 170 W. A complete desktop with an RTX 3060, a typical 6–8-core CPU, two NVMe drives, and a couple of case fans pulls 60–90 W at desktop idle and 200–280 W under inference — corroborated by 2026 r/LocalLLaMA owner reports linked in community threads referenced in the Open-WebUI repo.
The 24/7 power math (rounded, US average electricity ~$0.16/kWh as of 2026):
| Topology | Average draw | Annual kWh | Annual cost |
|---|---|---|---|
| GPU box always on | ~75 W | 657 | ~$105 |
| Pi always on + GPU box on-demand (~3 h/day active, ~$0.05/h active uplift) | ~5 W base + ~75 W * 3 h/day | 124 | ~$20 |
| Pi always on + GPU box wake-on-LAN, sleeping otherwise | ~5 W base + brief wake | 50–80 | ~$8–13 |
Even the conservative scenario saves roughly $85/year, and the wake-on-LAN scenario closer to $90/year. Over a 3-year lifecycle that pays for the Pi, the BX500 SSD, and the cables several times over.
Common pitfalls and gotchas
- Forgetting
OLLAMA_HOST=0.0.0.0. Ollama's default localhost bind is the single most common reason the Pi sees an empty model list. The Ollama FAQ networking section walks through the systemd drop-in for Linux, thelaunchctlflag for macOS, and the environment variable for Windows — set it once and the rest of the stack works. - Trusting microSD for the chat database. Even nominally "high-endurance" microSD cards fail under Open-WebUI's write pattern faster than people expect. Mount the Crucial BX500 1TB SATA SSD and bind-mount the data directory; recovering a corrupted SQLite file mid-conversation is no fun.
- Running multi-user without auth. Open-WebUI's first registered user is admin and subsequent users default to standard accounts. If you expose the Pi outside the LAN without a reverse proxy + auth, anyone who finds it can drain your GPU box's tokens. Keep it on the LAN, on a VPN like Tailscale, or behind an authenticating reverse proxy.
- Wi-Fi for the GPU box. The 3060 host should be on wired Ethernet. Wi-Fi adds tens of milliseconds of jitter to the per-chunk streaming path and surfaces as visible stutter in the UI. Per the Raspberry Pi 4 product page, the Pi itself has true Gigabit, so wire both.
- Ignoring the model cache size. Ollama stores GGUFs in
~/.ollama/modelsby default. A serious model library easily hits 60–100 GB. Point it at the 3060 host's largest drive — not the Pi.
When NOT to use this split
This pattern is overkill if you only use a local LLM once a week. In that case, running Open-WebUI on the GPU workstation itself and waking the box manually is simpler. It is also wrong if the "GPU box" is a laptop that closes its lid — laptops drop network connections and confuse the proxy. Finally, if every household member wants concurrent chat sessions plus voice transcription plus image generation, the Pi proxy is fine but the 3060 itself becomes the bottleneck; per TechPowerUp's RTX 3060 specs the card's 12 GB and 360 GB/s memory bandwidth comfortably serve one to three concurrent 7–9B users but stretch thin beyond that.
Bottom line: the always-on, low-power LLM dashboard
As of 2026, the Pi-front-end-plus-GPU-back-end split is the cleanest home-lab architecture for self-hosted LLMs. The Raspberry Pi 4 Model B 8GB is overpowered for the Open-WebUI container's actual workload, which means it stays responsive even when the ZOTAC GeForce RTX 3060 12GB box behind it is busy generating tokens. A Crucial BX500 1TB SATA SSD on USB 3.0 protects the chat history and RAG index from microSD wear, and the power math saves roughly $85–$90 per year versus leaving the GPU desktop on around the clock.
The whole build comes down to three environment variables, one Docker container, and a wired LAN. Set OLLAMA_HOST=0.0.0.0:11434 on the GPU box, set OLLAMA_BASE_URL to that host's IP on the Pi, bind-mount the data directory to the SSD, and the rest is just picking models.
Related guides
- LLM-rig planning and per-card tok/s comparisons live on the SpecPicks GPU benchmark hub.
- For workstation cards versus consumer GPUs in inference, see SpecPicks coverage at /articles?vertical=ai-rigs.
- Storage tier picks for home-lab AI servers: see SpecPicks SSD coverage at /category/ssds.
Citations and sources
- Open-WebUI on GitHub — project README, architecture notes, and community discussions referenced for resource usage and RAG behavior.
- Raspberry Pi 4 Model B product page — board specs, Gigabit networking, and typical power-draw figures.
- Ollama FAQ (GitHub) — networking configuration (
OLLAMA_HOST),OLLAMA_BASE_URLclient behavior, and embedded-model guidance.
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
