Open WebUI on a Raspberry Pi 4, Inference on an RTX 3060: The Split-Brain Home LLM Stack

Short answer: Yes. The Raspberry Pi 4 Computer Model B 8GB makes an excellent always-on front-end and API gateway for Open WebUI, pointing at Ollama or llama.cpp running on a separate GPU host. Wake-on-LAN brings the GPU box up on first request; it sleeps between requests. Power savings are real and the setup is a weekend project — but only worth doing if your GPU box is loud, in a bedroom, or costs you real money at idle.

Step 0 — is the problem idle power, availability, or noise?

Before you build this, be honest about what problem you are solving. Three common ones, and only two of them are worth the split-brain setup:

• Idle power draw. A desktop with a discrete GPU typically pulls 50-100W at idle, 24/7. That is 400-800 kWh per year of doing nothing. If your electricity costs $0.15/kWh, that is $60-$120/year. This split saves most of that.
• Noise. A GPU box with three case fans in a bedroom or shared office is a persistent low-level annoyance. Sleeping it between requests moves the noise floor to zero.
• Always-on chat availability. You want to send a message from your phone and get a reply within a few seconds, without the GPU box needing to be on. The Pi answers first, then wakes the GPU.

The one problem this does not solve is chat throughput. If your GPU box is already on around the clock for other purposes (media server, homelab hypervisor, dev environment), the Pi adds complexity for no benefit. Skip the split-brain and run Open WebUI directly on the GPU host.

Why leaving a 200W+ GPU box awake around the clock is the wrong default

The homelab default has always been "leave the box on, deal with the electricity bill." That was fine when the box drew 40W and idled at 15W. Modern gaming/inference PCs with discrete GPUs pull 40-80W at idle even with sane power settings, and that is before the display, monitor, or peripherals wake up. Across a year, that is $60-$120 of electricity, several tons of heat dumped into the room, and continuous fan noise.

The alternative is to make the always-on part very small — a Pi drawing 3-6W — and let the big box sleep. First request wakes the GPU host over the network. First-token latency goes up by 15-40 seconds on a cold start, then drops to normal. For non-time-critical chat, that is a fine trade.

Key Takeaways

• Pi 4 idle: ~3-6W depending on peripherals attached; RTX 3060 desktop idle: 50-100W depending on platform and power settings.
• Wake-on-LAN latency: 15-40 seconds from magic packet to a responding inference server, depending on boot path and model preload.
• Monthly kWh delta: roughly 30-60 kWh if the GPU box otherwise idles 24/7 and now sleeps ~20 hours per day. At $0.15/kWh, that is $5-$9 per month or $60-$108 per year.
• Pi 4 payback period: 2-6 months in electricity savings alone, depending on your rates and idle power.
• What the Pi handles: UI, auth, RAG document storage, chat history, wake-on-LAN, request routing. It does not run the model.

What you'll need

• Always-on gateway: Raspberry Pi 4 Computer Model B 8GB. 8GB is comfortable for Open WebUI + a small database + a reverse proxy. The 4GB SKU works too but leaves less room for browser tabs on the same box if you use it for anything else.
• Storage for chat history + RAG docs: Crucial BX500 1TB SATA SSD connected via a USB 3.0 adapter. Chat history is small; RAG document embeddings and their source files are what actually consume space. 1TB is generous.
• GPU host: any desktop with a modern discrete GPU. The MSI GeForce RTX 3060 Ventus 3X 12G is the reference for this piece — 12GB is enough for 8B at full precision or 14B at q4.
• Wired network path. Ethernet is required. Wake-on-LAN over Wi-Fi does not reliably work.
• Optional: a Raspberry Pi Zero W Basic Starter Kit as a network-presence probe on the far side of your network, if you want to trigger the GPU box from a location the Pi 4 cannot reach directly.

Architecture walkthrough

Three tiers:

1. Front-end (Pi 4): Open WebUI runs in a Docker container. It handles the browser UI, user authentication, chat history storage, and RAG document management. LiteLLM (or Open WebUI's built-in OpenAI-compatible router) sits alongside to normalize the API surface. A wake-on-LAN helper script watches for outgoing inference requests and issues the magic packet if the GPU host is asleep.
2. Inference tier (GPU host): Ollama or llama.cpp exposes an OpenAI-compatible API on the local network. The GPU host runs a light idle-shutdown script that puts the machine to sleep after N minutes of inactivity on the inference port.
3. Reverse proxy (Pi 4): Caddy or nginx sits in front of both. Requests to /api/chat route through the wake-on-LAN helper, wait for the GPU host to come up, then forward. This adds latency to the first request in a session and none to subsequent ones.

The wake helper is the interesting piece. When Open WebUI issues a request to the inference API, the proxy first pings the GPU host. If it responds, forward the request. If not, send a magic packet, wait 15-40 seconds for boot+model preload, then forward. Downstream users see a spinner for a longer first response and normal latency thereafter.

Step-by-step install (real commands)

The following is skeletal — adapt to your OS and network layout. Runs on Raspberry Pi OS Bookworm or Ubuntu Server 24.04 for ARM.

# On the Pi:
sudo apt update && sudo apt install -y docker.io docker-compose wakeonlan
sudo usermod -aG docker $USER
newgrp docker

# Open WebUI + LiteLLM stack
mkdir -p ~/llm-gateway && cd ~/llm-gateway
cat > docker-compose.yml <<'YAML'
services:
 open-webui:
 image: ghcr.io/open-webui/open-webui:main
 ports: ["3000:8080"]
 volumes: ["./data:/app/backend/data"]
 environment:
 - OPENAI_API_BASE_URL=http://litellm:4000/v1
 - OPENAI_API_KEY=sk-local
 restart: unless-stopped
 mem_limit: 3g
 litellm:
 image: ghcr.io/berriai/litellm:main-latest
 volumes: ["./litellm.yaml:/app/config.yaml"]
 command: ["--config", "/app/config.yaml"]
 restart: unless-stopped
 mem_limit: 512m
YAML

# Point LiteLLM at the (asleep-for-now) GPU host
cat > litellm.yaml <<'YAML'
model_list:
 - model_name: qwen3-14b
 litellm_params:
 model: openai/qwen3:14b
 api_base: http://gpu-host.local:11434/v1
 api_key: none
YAML

docker compose up -d

On the GPU host, Ollama runs on port 11434 with the model preloaded on wake. Idle-shutdown script sleeps the machine after ~15 minutes of no inference traffic. On the Pi, a small wake-on-request.sh sits in front of nginx and sends wakeonlan <mac> when the GPU host is unreachable. The Pi documentation at raspberrypi.com covers systemd unit setup and USB storage mounting.

Wake-on-LAN and idle-shutdown mechanics

Wake-on-LAN setup on the GPU host:

1. BIOS: enable "Wake on PCI-E" or "Wake on LAN" — the specific menu name varies by board.
2. OS: on Linux, ethtool -s eth0 wol g; on Windows, in Device Manager → NIC properties, enable "Wake on Magic Packet."
3. Test from the Pi: wakeonlan AA:BB:CC:DD:EE:FF should bring the box up from a soft-off state.

Idle-shutdown script (Linux GPU host):

#!/bin/bash
# /usr/local/bin/idle-sleep.sh — check every 60s
while true; do
 sleep 60
 # If no inference traffic in the last 15 min, sleep
 if [ -z "$(ss -tn state established '( sport = :11434 )')" ]; then
 IDLE_COUNT=$((IDLE_COUNT + 1))
 else
 IDLE_COUNT=0
 fi
 if [ "$IDLE_COUNT" -ge 15 ]; then
 systemctl suspend
 IDLE_COUNT=0
 fi
done

Wire it as a systemd service. Adjust the idle threshold to your tolerance — 15 minutes is a reasonable default.

Expected wake times:

• Suspend-to-RAM → responsive: 5-10 seconds. Model still in VRAM (best case).
• Suspend-to-disk / hibernate → responsive: 20-40 seconds. Model has to reload.
• Fully powered off → responsive: 40-90 seconds. Full boot path.

Configure Open WebUI to show a "waking backend" state on first request rather than timing out the connection.

Benchmark: Pi-proxied versus direct

Rough measurements on the reference setup (Pi 4 8GB + RTX 3060 12GB, gigabit wired ethernet between them, GPU host awake for both tests):

The proxy overhead is negligible once the GPU host is awake — the Pi is fast enough to relay OpenAI-compatible HTTP traffic without becoming the bottleneck. The bottleneck is boot time on a cold start, not steady-state throughput.

Power math

Assumptions: GPU box idle at 65W, active at 200W. Pi at 5W constant. User makes 20 chat requests per day, GPU box active for ~15 minutes total per day, sleeps otherwise.

• Old setup (GPU box always on): 65W × 24h × 30d = 46.8 kWh/month
• New setup: (Pi: 5W × 24h × 30d) + (GPU active: 200W × 0.25h × 30d) + (GPU idle: 65W × 0h) = 3.6 + 1.5 = 5.1 kWh/month
• Savings: ~42 kWh/month at those assumptions. At $0.15/kWh, ~$6.30/month or $75/year.

Your numbers will vary. Measure with a Kill-A-Watt-class inline meter before and after to get real values. If your GPU box idles closer to 100W (older platforms, higher-end GPUs), the savings roughly double. If you use the box heavily for other things (gaming, dev work) so it is only asleep 4-6 hours a day, the savings shrink accordingly.

Where this breaks down — the honest counter-case

Skip this setup if:

• The GPU box is already on 24/7 for other reasons. Media server, homelab hypervisor, always-on dev environment. Adding the Pi buys you nothing.
• You value very low first-token latency on every request. A 15-second wake is a papercut for the first message in a session.
• You share the setup with users who will not tolerate the wake pattern. Kids, spouse, coworkers who expect chat to feel like a hosted API.
• You have no Ethernet path to the GPU host. Wake-on-LAN over Wi-Fi is not reliable enough for this to work.

Complete-the-build sidebar

Cooling the GPU host — a set of ARCTIC P12 PWM PST 5-pack case fans in intake/exhaust configuration keeps a 3060 or 3060 Ti below thermal throttle under sustained inference load. The stock cooling on most cases is not sized for 30 minutes of GPU-pinned work.

Network-presence probe (optional) — a Raspberry Pi Zero W Basic Starter Kit parked in a different subnet can ping the GPU host on your behalf and issue wake packets from a location the Pi 4 cannot directly reach. Overkill for a single-room home lab; useful in a multi-VLAN setup.

Bottom line

For the specific case of "GPU box in a bedroom/office where noise and idle power matter," this split-brain arrangement is the right move. Weekend to set up, meaningful electricity savings, quieter home, and Open WebUI feels the same as if the whole stack ran on one machine — with a longer wait on the first message of a session. For everyone whose GPU box is already always-on for other purposes, skip the complexity and run Open WebUI directly on the GPU host.

The Pi is one of the most versatile pieces of hardware you can own — this is one of the higher-leverage uses of a spare unit, and it pairs well with the other services (Pi-hole, HomeAssistant, small monitoring dashboards) that a Pi 4 8GB comfortably runs in parallel. Once you have the wake-on-request pattern working for the LLM stack, the same design applies to anything else you would rather leave asleep — a beefy build server, a media transcode host, a machine-learning training rig. The Pi becomes your always-on control plane.

Related guides

• Inkling 975B Open Weights: What a Frontier Speech Model Means for Local Rigs
• Open-Weight Models Caught Up on Cyber Benchmarks in Four Months
• Raspberry Pi 5 IOMMU Support Is Landing in the Mainline Linux Kernel

Citations and sources

• Open WebUI documentation — installation, feature reference, OpenAI-compatible API configuration (accessed July 2026).
• Raspberry Pi — official documentation — Pi 4 hardware, systemd setup, USB storage mounting (accessed July 2026).
• llama.cpp on GitHub — inference server reference and OpenAI-compatible HTTP mode (accessed July 2026).

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.