Halloween Talking Prop: Qwen3 4B on Jetson Orin Nano Super vs Raspberry Pi 5 16GB

The offline haunted-house prop

The build is a skeleton, a witch or a talking pumpkin that hears a trick-or-treater and answers in character. Inside, five stages run in a loop. A USB mic captures the kid's question. A speech-to-text model such as OpenAI Whisper turns it into text. A small language model writes a one- or two-sentence reply in the character's voice. A local text-to-speech engine such as Piper speaks it. Then a powered speaker, and maybe a servo jaw, plays it back.

Why run it all on the board instead of calling a cloud API? Porches have bad Wi-Fi. A cloud round-trip adds latency you can't control. And if the router drops at 7:30 p.m., a cloud-backed prop goes silent at the busiest point of the night. An on-device stack has no network dependency, no per-request cost and no audio leaving the yard. That makes the board's own inference speed the whole question.

Qwen3 4B is a sensible default brain for the job. Per its model card, it has 4.0B parameters (3.6B non-embedding), 36 layers and grouped-query attention with 32 query and 8 KV heads. It supports 32,768 tokens of context natively and 131,072 with YaRN. Qwen ships an official GGUF build whose Q4_K_M file is 2.50 GB, small enough for either board.

This piece is synthesis: every speed figure below comes from a published benchmark linked inline or from the aggregated medians on SpecPicks' benchmark pages.

Key Takeaways

• The GPU sets the gap. Per the Turing Pi Orin Nano benchmark, Qwen3 4B Q4_K_M generates 19.53 tok/s on the Orin Nano Super in its 25W mode, with a 1.26 s time to first token.
• The Pi 5 has no published Qwen3 4B run. Other 3.8-4B Q4_K_M models land between about 3 and 9.5 tok/s on the Pi 5 in community measurements. That is enough for short lines, not for banter.
• Prompt processing matters more than generation. The Orin Nano Super processes the prompt for Qwen3 4B at 609.7 tok/s per Turing Pi. For 3B models on the Pi 5, pi5-llm-arena logged 28.89 tok/s in llama.cpp. A long character prompt costs seconds on the Pi and fractions of a second on the Jetson.
• Launch prices are $249 vs $120. The Super price comes from NVIDIA's announcement. The 16GB Pi price comes from the Raspberry Pi launch post.
• Memory isn't the problem on either board. Turing Pi recorded 2,913 MiB of memory use for Qwen3 4B Q4_K_M on the 8GB Jetson. That leaves room for a small Whisper model and a Piper voice.

Step 0: How fast does a talking prop actually need to be?

A prop doesn't need chatbot speed. It needs a reply that starts before the kid walks to the next house. Three stages add up to that delay:

1. Speech-to-text. Whisper's README lists the tiny (39M parameters) and base (74M) models at about 1 GB of VRAM each, and the small model (244M) at about 2 GB. Tiny and base are the realistic picks on an SBC, and the whisper.cpp port is the usual way to run them on Arm and Jetson boards.
2. Language model. This stage has two parts: processing the prompt (persona instructions plus the transcribed question), then generating the reply.
3. Text-to-speech. Piper is built for small local devices and can start speaking the first sentence while the model is still writing the second.

Streaming is what makes a slow board usable. If the LLM streams tokens and you hand each finished sentence to Piper, the kid hears speech as soon as the first sentence exists. So the number that matters is time to the first complete sentence, not time to the whole reply.

A one-sentence line like "Who dares to knock on my crypt tonight?" is roughly 10-15 tokens. At the Pi 5's 1-4B median of 7.5 tok/s that's about 2 seconds of generation. At the Orin Nano Super's Qwen3 4B rate of 19.53 tok/s it's well under one second. Both are fine for a single creepy line. The difference shows up in three places: long persona prompts, multi-sentence replies, and conversations where the prop has to remember what the kid said earlier.

When a 1-2B model is enough: if the prop only needs to pick a canned theme and riff one sentence on it, Qwen3 1.7B is the better match. Turing Pi measured it at 39.42 tok/s generation and 1,408.3 tok/s prompt processing on the Orin Nano Super. The OpenClaw Pi deployment notes settled on Qwen3 1.7B Q4_K_M as the primary model on a Pi 5 8GB after the 4B model proved unstable under load. Same SoC, same lesson.

Spec delta table

Sources: Jetson figures and price from NVIDIA's Super-boost post. Pi 5 clock from the Tom's Hardware Pi 5 16GB review, price from the Raspberry Pi 16GB launch post. Pi 4 8GB price from the Raspberry Pi 4 8GB launch post. The Super upgrade raised the module's memory bandwidth from 68 GB/s to 102 GB/s and added the 25W mode. That bandwidth is the main reason token generation sped up, since generating each token means streaming the model's weights from memory.

The developer kit is sold under several packages. The Waveshare Jetson Orin Nano Super AI Development Kit and the Yahboom Jetson Orin Nano 8GB Super board both carry the same 8GB Orin Nano module. On the Pi side, this guide means the Raspberry Pi 5 (16GB) and, as the budget floor, the Raspberry Pi 4 Model B 8GB.

Qwen3 4B per-quant table

File sizes come from Qwen's official Qwen3-4B-GGUF repository. The BF16 size is derived from the 4.0B parameter count on the model card at 2 bytes per weight. "Fits" means the weights fit with room left for a short context and the speech stack. It is not a claim that anyone ran that combination.

For a prop, Q4_K_M is the right call on both boards. Generation speed on both is limited by memory bandwidth, so a bigger file generates more slowly. A slightly better word choice from Q6_K or Q8_0 isn't worth the slower reply when the line is "Muahaha, take two."

How many tokens per second does each board really get?

These are medians across every published 1-4B run in SpecPicks' benchmark database, with the per-run sources listed on each benchmark page:

The median mixes 1B and 4B models and several runtimes (llama.cpp, Ollama, MLC). So it is a class figure, not a Qwen3 4B figure. The one direct Qwen3 4B measurement on the Jetson, 19.53 tok/s, is about half the class median, which is what you'd expect for a model at the top of the size range.

For the Pi 5, the closest stand-ins are other ~4B models at Q4_K_M:

• Phi-3.5 3.8B: 7.4 tok/s per LocalAI Master.
• Gemma 3 4B: two runs at 3.5 and 9.5 tok/s per Kunal Ganglani's Pi 5 benchmark.
• Phi-4-mini 3.8B: 3.0 tok/s in the OpenClaw Pi notes.
• Llama 3.2 3B: 4.51-4.88 tok/s in Jeff Geerling's ai-benchmarks.

A fair expectation for Qwen3 4B on the Pi 5 is the 3-9 tok/s band. Treat that as an inference from neighboring models, not a measurement.

The Pi 4 has no 1-4B runs in the database. The nearest public figure is the Pi 400, which uses the same BCM2711 chip as the Pi 4. Per Jeff Geerling's benchmark commit, it runs Llama 3.2 3B at 1.60 tok/s. That rules the Pi 4 out for a 4B conversational prop.

Does context length or the speech stack eat the memory?

Prompt processing is the hidden cost. Every reply begins by processing the system prompt (character name, backstory, rules like "never break character, keep it under 20 words") plus the transcribed question. A 300-token persona prompt works out like this:

• Orin Nano Super: about half a second at Turing Pi's 609.7 tok/s for Qwen3 4B.
• Pi 5: roughly 10 seconds at the 28.89 tok/s that pi5-llm-arena logged for a 3B model in llama.cpp. Ollama runs in the same repository logged faster prompt rates on 3B models (178.39 tok/s for Qwen2.5 3B), so runtime and caching matter a lot.

The practical fix on either board is to keep the persona prompt short and let the runtime cache it between turns. llama.cpp and Ollama both reuse the KV cache for an unchanged prompt prefix.

Longer context costs speed and memory. Turing Pi pushed Qwen3 4B to a filled 16k-token context on the Orin Nano Super. Prompt processing fell to 436.0 tok/s and generation to 12.03 tok/s. The run failed at 32k when it tried to allocate a larger FP16 KV buffer. A prop that forgets everything after each kid needs a few hundred tokens of context, not 16k. Cap num_ctx / -c at 2,048-4,096.

The speech stack fits. On the 8GB Jetson, Turing Pi's 2,913 MiB figure for Qwen3 4B, plus about 1 GB for Whisper tiny or base per the Whisper README, plus a Piper voice and the OS, still leaves headroom. On the Pi 5 16GB, memory isn't a constraint at all. The OpenClaw notes give a useful warning for 8GB Pis, though: Qwen3 4B at an 8,192 context allocated about 4.2 GB, which left only about 1.5 GB of headroom and caused hangs under load. The 16GB Pi avoids that failure mode entirely.

Which board for which prop?

Budget: Raspberry Pi 4 Model B 8GB. Best for scripted props. Have it handle a PIR motion sensor, a servo jaw and a library of pre-recorded or pre-generated lines. If you want some variety, run Qwen3 0.6B or 1.7B to pick and lightly remix a line, and keep each reply to one sentence. Given the Pi 400's 1.60 tok/s on a 3B model, don't ask it to hold a conversation.

Mid: Raspberry Pi 5 (16GB). Runs Qwen3 4B at Q4_K_M for slow, short replies. Expect a pause of a couple of seconds before each answer, which a spooky character can sell ("Hmmmm... who goes there?"). Pair it with Whisper tiny through whisper.cpp and a Piper voice. If the pauses feel too long, step down to Qwen3 1.7B.

Performance: Jetson Orin Nano Super kit. Choose the Waveshare kit or the Yahboom Super board. This is the real-time option: Qwen3 4B starts its reply in about 1.3 seconds and generates near 20 tok/s, so back-and-forth conversation feels natural. Whisper can also run on the GPU, which shortens the speech-to-text stage.

Power, heat and weatherproofing on a porch for one night

• Power. Per NVIDIA, the Super module runs in 7W, 15W or 25W modes. The Turing Pi measurements above were taken in the 25W profile, so run the bundled barrel-jack adapter and select that mode. For the Pi 5, Raspberry Pi sells an official 27W USB-C power supply (5.1V/5A). Weaker phone chargers trigger undervoltage warnings and throttling, and that will show up as slower replies mid-evening.
• Heat. Sustained inference keeps the CPU cores (Pi) or GPU (Jetson) under steady load for hours. The Jetson kit ships with an active fan heatsink. On the Pi 5, add the official Active Cooler or a fan case. pi5-llm-arena logged peak temperatures around 56-58°C on cooled 1.5-7B runs.
• Enclosure mistakes buyers often make:
• Sealing the board in a plastic pumpkin with no airflow.
• Running an indoor power strip in the rain.
• Putting the mic right next to the speaker, so the prop hears itself and answers its own lines.

Vent the enclosure from below, shield it from rain from above, use an outdoor-rated extension cord, and mount the mic away from the speaker grille.

• Storage. A cheap microSD card can corrupt if power drops. Use a good card or an NVMe drive, and make the prop's service start automatically at boot so a power blip recovers on its own.

Perf-per-dollar and perf-per-watt math

Price basis is launch MSRP only: $249 for the Orin Nano Super Developer Kit (NVIDIA) and $120 for the Pi 5 16GB (Raspberry Pi). Third-party kits like the Waveshare and Yahboom bundles are priced differently; check the live price buttons on this page. Power basis is the rated power budget (Jetson's 25W mode; the Pi's 27W supply rating). These are ceilings, not measured wall draw, so the per-watt column compares budgets, not efficiency measured at the wall.

Even with the Pi's lower price, the Jetson delivers about 2.5× the tokens per dollar on this class of model. It also does far more work per watt of budget. The Pi 5 wins only if you already own one, or if the prop truly needs nothing more than one short line per visitor.

Common pitfalls

1. Qwen3's thinking mode. Qwen3 models think out loud by default. The OpenClaw notes recorded Qwen3 4B spending 147+ internal reasoning tokens on a simple request, using up its output budget before any visible reply. On a prop, that's a long silence. Disable thinking (think: false in Ollama, or /no_think in the prompt) for every request.
2. Context set too high. The default context in some front ends allocates far more KV cache than a prop needs. Cap it at 2,048-4,096 tokens.
3. Feedback loops. If speech-to-text hears the prop's own speaker, the prop answers itself. Mute the mic while TTS plays.
4. The 7W/15W default on the Jetson. Published Jetson numbers assume the 25W mode. A kit left in a lower mode will undershoot them.

When NOT to build this

If your prop plays the same three lines all night, skip the LLM entirely. A Pi 4 with a motion sensor and pre-rendered Piper audio is cheaper, cooler and nearly impossible to break. And if the porch has no power outlet, neither board is a good fit for a night on a battery pack without careful power planning.

Verdict matrix

• Get the Raspberry Pi 5 16GB if… you want an open-ended Qwen3 4B prop under the lowest launch price, you're fine with two- to five-second pauses, and you'll keep replies to one sentence.
• Get the Jetson Orin Nano Super if… you want real back-and-forth: a reply that starts in about 1.3 seconds and generates near 20 tok/s on Qwen3 4B, with GPU headroom for Whisper too.
• Stick with the Raspberry Pi 4 8GB if… you already own one and the prop mostly plays scripted lines, optionally picked or remixed by a sub-2B model.

Recommended pick: for a prop built this week for Halloween 2026, the Jetson Orin Nano Super is the board that makes an LLM prop feel alive rather than laggy. The 19.53 tok/s Qwen3 4B measurement and the 1.26 s time to first token are the numbers that matter. If budget is the constraint, a Pi 5 16GB running Qwen3 1.7B is the honest fallback.

Related guides

• Local AI on a Raspberry Pi in 2026: what actually runs
• Jetson Orin Nano Super vs Raspberry Pi 5: edge AI benchmarks
• Jetson Orin Nano Super local LLM tokens per second
• Best hardware for the Qwen3 family of local LLMs
• Qwen3 4B on a Raspberry Pi 4 8GB vs Ryzen 5 5600G
• Raspberry Pi 5 power supplies
• The other Halloween build: FLUX.2 Klein 4B on RTX 3060 12GB vs RTX 5060 8GB

Live price comparison

Compare the Raspberry Pi 5 16GB and the Waveshare Jetson Orin Nano Super kit side by side for current prices and Amazon links. Prices may vary; the live widget shows the latest figure.

Citations and sources

Accessed 2026-10-19.

• NVIDIA Technical Blog — Jetson Orin Nano Developer Kit Gets a Super Boost
• Raspberry Pi — 16GB Raspberry Pi 5 on sale now at $120
• Raspberry Pi — 8GB Raspberry Pi 4 on sale now at $75
• Raspberry Pi — 27W USB-C Power Supply
• Tom's Hardware — Raspberry Pi 5 16GB review
• Hugging Face — Qwen/Qwen3-4B model card
• Hugging Face — Qwen/Qwen3-4B-GGUF
• Turing Pi — Jetson Orin Nano 8GB LLM benchmark
• OpenClaw Pi OSS — Ollama model research (Pi 5)
• pi5-llm-arena (GitHub)
• LocalAI Master — LLMs on Raspberry Pi 5
• Kunal Ganglani — Gemma 3 Raspberry Pi 5 benchmark
• geerlingguy/ai-benchmarks — issue #7
• geerlingguy/ai-benchmarks — Pi 400 results commit
• OpenAI Whisper (GitHub)
• whisper.cpp (GitHub)
• Piper TTS (GitHub)

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.