Local AI on a Raspberry Pi in 2026: What Actually Runs (and What Doesn't)
Pi 4, Pi 5 (8GB/16GB) and the AI HAT+ 2: measured tok/s for Qwen3, Gemma 3, Llama 3.2 and a 30B MoE, and what to skip.
By Mike Perry, Founder & Editor-in-Chief · Published 2026-05-29 · Updated 2026-10-16 · 10 min read
As of Oct 2026 a Pi 5 runs Qwen3 0.6B at 16.6 tok/s and Gemma 4 E2B at 9; a $305 16GB board runs a 30B MoE at 8 tok/s.
Quick Answer
As of October 2026, a Raspberry Pi 5 8GB runs the 14 MB Needle function-calling model on its CPU at 248–314 tok/s decode and turns "Turn the LED on" into a function call in 78 ms, per Raspberry Pi's September 22, 2026 Needle post, so tiny task-specific models are effectively instant. For general chat, a Raspberry Pi 5 decodes Llama 3.2 3B at 5.71 tok/s CPU-only under llama.cpp, per the pi5-llm-arena benchmark, while a Jetson Orin Nano 8GB in its 25 W mode runs the same 4-bit model at 25.24 tok/s, per Turing Pi's llama-bench results, about 4.4× faster. Sub-1B models are where the Pi is comfortable: about 16.6 tok/s on Qwen3 0.6B and 10 tok/s on Gemma 3 1B under Ollama, per Adafruit's Pi 5 Qwen3 and Gemma 3 runs. A 16GB Pi 5 can also run the Qwen3 30B-A3B mixture-of-experts model at 7.99 tok/s using ByteShape's 2.70-bit quant, per Geerling's llama-bench run. The $130 AI HAT+ 2 (Hailo-10H) is slower than the Pi's own CPU for LLMs: 6.74 vs 11.73 tok/s on Qwen2.5 1.5B, per CNX Software's review. Google's Gemma 4 E2B decodes at 9 tok/s on a Pi 5 8GB under LiteRT-LM (7 tok/s under llama.cpp), per Raspberry Pi's LiteRT write-up. The newest NPU figure repeats the AI HAT+ 2 pattern: Sixfab's 25 TOPS AI HAT+ (DEEPX DX-M1M) runs Qwen3 1.7B at 4.64 tok/s on a Pi 5 8GB, per Raspberry Pi's October 5, 2026 post. Vision and audio are still where the Pi shines.
What changed — October 2026 refresh
October 16, 2026: tiny function-calling models are the new fast tier. Per Raspberry Pi's Needle post (September 22, 2026), the 14 MB Needle model ran CPU-only on a Pi 5 8GB at 461–488 tok/s prefill and 248–314 tok/s decode, with whole requests completing in 76–149 ms and the process peaking at 43–46.4 MB. It picks a function and its arguments; it is not a chat model.
October 16, 2026: a second NPU HAT measured on LLMs. Per Raspberry Pi's October 5, 2026 post, the Sixfab AI HAT+ (DEEPX DX-M1M NPU, 25 TOPS at about 3 W, 2 GB on-module LPDDR4X) ran Qwen3 1.7B at 4.64 tok/s with a 599 ms time to first token on a Pi 5 8GB. The Pi 5's own CPU runs the same model at 6.89 tok/s under Ollama, per the OpenClaw log, though the two runs use different runtimes and quants.
Gemma 4 E2B now has a published Pi 5 figure. Per Raspberry Pi's August 2026 LiteRT + Gemma post, Gemma 4 E2B (QAT, 4 threads, Pi 5 8GB) decodes at 9 tok/s and prefills at 99 tok/s under LiteRT-LM with a 1,432 MB peak memory footprint, against 7 tok/s decode, 58 tok/s prefill and 4,406 MB peak under llama.cpp. It replaces the earlier "no verifiable figure" note for Gemma 4.
The 16GB Pi 5 is now a $305 board. Raspberry Pi's product page lists the 16GB model at $305, up from $120 at launch; the February 2026 memory-driven price rise alone added $60 to it. A Raspberry Pi forum price summary records the 8GB at $175 since April 1, 2026. That changes the value math against a Jetson Orin Nano Super (see below).
A four-Pi cluster runs Qwen3 30B-A3B at about 15 tok/s. Per LinuxGizmos (October 6, 2026) and the underlying Zenodo paper, a modified distributed-llama build on four Pi 5 16GB boards reached 15.143 tok/s decode versus 13.15 tok/s for unmodified distributed-llama on the same hardware. The author notes the result has not yet been independently reproduced.
Re-verified, unchanged: the Adafruit Qwen3 0.6B (16.62 tok/s) and Gemma 3 1B (9.99 tok/s) runs, Geerling's Llama 3.2 3B rows, and his Qwen3 30B-A3B 16GB result (7.99 tok/s) were rechecked against their source pages and still stand.
Deeper pass (October 13, 2026): a same-model, three-board table. A new cross-board table (below the Pi 5 model table) puts the Pi 5 CPU, the AI HAT+ 2 and a Jetson Orin Nano side by side on identical models. Llama 3.2 3B runs at 5.71 tok/s on a Pi 5 16GB under llama.cpp (pi5-llm-arena), 2.60 tok/s on the Hailo-10H (CNX Software) and 25.24 tok/s on the Jetson (Turing Pi).
Qwen3 1.7B added. It runs at 6.89 tok/s on a Pi 5 8GB under Ollama at q4_K_M, per an OpenClaw-on-Pi measurement log, against 34.16 tok/s under Ollama and 39.42 tok/s under llama.cpp on the Jetson, per Turing Pi.
Gemma 4 E2B cross-checked. A second Pi 5 source measured 7.04 tok/s under llama.cpp (pi5-llm-arena), matching Raspberry Pi's 7 tok/s figure. The Jetson runs the same Q4_K_M model at 26.67 tok/s (Turing Pi).
8B-class rows added. Llama 3.1 8B decodes at 2.46 tok/s on a Pi 5 16GB under llama.cpp (pi5-llm-arena). Qwen3 8B runs at 12.64 tok/s on the Jetson, per Turing Pi, and NVIDIA lists Llama 3.1 8B at 19.14 tok/s on the Orin Nano Super under MLC (NVIDIA).
What had changed by September 2026
Three things moved: sub-1B Qwen3 and Gemma 3 models run 2–3× faster than the old 3B defaults; MoE models let a 16GB Pi 5 run a 30B model because only ~3B parameters are read per token; and the AI HAT+ 2 launched January 15, 2026 at $130 (Hailo-10H, 40 TOPS INT4, 8 GB on-board RAM).
All figures below are community or reviewer measurements of decode speed. File sizes are the Hugging Face GGUF for the named quant. "Ceiling" is the Pi 5's ~17 GB/s bandwidth divided by bytes read per token.
Every measured figure sits at 40–80% of its ceiling, so claims of 20+ tok/s for a 3B model on a Pi 5 are suspect. The 30B MoE file (10.3 GB) only fits the 16GB board; Geerling says it "barely fits (with context)", and ByteShape's write-up reports a consistent 8.03 tok/s. The CNX CPU figure comes from a Compute Module 5, which shares the Pi 5's BCM2712 SoC and LPDDR4X memory. Gemma 4 E2B's figure comes from Raspberry Pi's own LiteRT write-up; its "ceiling" is left blank because the runtime's in-memory footprint (1,432 MB peak) differs from the file size. Qwen3.5 small models are still left out: no Pi 5 decode figure traceable to a named, reproducible run was found as of October 2026.
Same model, three boards: Pi 5 vs AI HAT+ 2 vs Jetson Orin Nano (October 2026)
The tables above compare models on one board. This one holds the model fixed and changes the board, which is the comparison that answers "should I buy a Pi for this?" All figures are single-stream decode at 4-bit. "—" means no traceable public run was found, not that the model fails.
The Jetson's lead tracks memory bandwidth. NVIDIA lists 102 GB/s for the Orin Nano Super against 68 GB/s for the original kit, per its Super announcement. That is roughly 6× the Pi 5's ~17 GB/s, and the measured like-for-like gaps above land at 3.8–5×.
The 25 W mode is the Super-era power mode. NVIDIA's Super announcement lists 7 W and 15 W modes for the original kit and adds 25 W for the Super. Turing Pi's runs used an Orin Nano 8GB module in that 25 W mode under JetPack 7.2.1 with llama.cpp llama-bench (tg128, five repetitions), per its write-up.
Value has flipped. At the Jetson Orin Nano Super Developer Kit's $249 launch price (NVIDIA, December 2024) against $305 for a Pi 5 16GB (Raspberry Pi), the cheaper board decodes Llama 3.2 3B about 4.4× faster. The Pi 5 16GB's remaining edge is capacity: only it fits the 10.3 GB Qwen3 30B-A3B quant covered above, because the Jetson has 8 GB.
Runtime still matters on the Pi. The same pi5-llm-arena run measured 5.92 tok/s for Llama 3.2 3B under Ollama and 5.71 under llama.cpp, while Geerling's Ollama run measured 4.61. Compare figures from one source and runtime where possible.
Caveats. The pi5-llm-arena README lists file sizes (1.88 GB for Llama 3.2 3B, 1.04 GB for DeepSeek R1 1.5B, 4.58 GB for Llama 3.1 8B) but does not name the quant. Each Pi figure for those three stays below the ~17 GB/s bandwidth ceiling for its file size. Gemma 4 E2B is left out of that check: the README lists its file at 6.67 GB, but Raspberry Pi measured a 1,432 MB peak footprint under LiteRT-LM, so file size does not bound its per-token reads. The OpenClaw Qwen3 1.7B run had thinking mode enabled, which affects wall time, not per-token speed.
Why the Raspberry Pi keeps showing up in "local AI" conversations
The Raspberry Pi Foundation has expanded what "local AI on a Pi" means: the Pi 5 (now up to 16GB) roughly tripled measured LLM throughput over Pi 4-class boards, and Hailo accelerators on its PCIe bus went from the 13 TOPS AI Kit to the 13/26 TOPS AI HAT+ and the 40 TOPS AI HAT+ 2. The cheap Pi 4 8GB is still what newcomers reach for when they hear "run an LLM at home."
The answer to "what can I run on a Pi?" depends on three numbers: DRAM, memory bandwidth, and whether a coprocessor (Hailo, Coral, or the AI Camera's on-sensor NPU) does the heavy lifting. CPU clock speed barely matters. Even the Pi 5's quad Cortex-A76 at 2.4 GHz is bandwidth-bound on transformer generation, not compute-bound.
Key takeaways
A Pi 5 runs current sub-1B models at 10–17 tok/s, conversational for short replies (figures above).
3B dense models run ~3× faster on a Pi 5 than a Pi 4-class board, but still slowly (4.61 vs 1.60 tok/s, per Geerling).
The 16GB Pi 5 is the board to buy for LLMs, but it now costs $305. Only it fits a 30B-A3B MoE, which handles tasks the tiny models fail; at its current list price a $249 Jetson Orin Nano Super is the cheaper step-up for pure LLM work.
The AI HAT+ 2 is for vision and low-power mixed workloads, not faster chat. The Pi's CPU beat the Hailo-10H on LLMs in both independent reviews cited below.
Object detection, wake-word and transcription are where the Pi excels. A Hailo accelerator runs YOLO roughly 10× faster than the Pi's CPU, per Geerling.
Anything 7B-and-up dense is masochism on a Pi. A Jetson Orin Nano Super or a Mac mini is far better for similar money.
Storage is the most-skipped variable. An NVMe HAT or a USB 3.0 SSD beats an SD card for model loading.
Hardware reality: what each Pi actually delivers in 2026
Board
CPU
Memory BW
Max RAM
Accelerator
Notes
Pi Zero 2 W
4× A53 @ 1 GHz
1.6 GB/s
512 MB
—
Vision only
Pi 4 8GB
4× A72 @ 1.5 GHz
~4 GB/s effective
8 GB
—
Sub-3B LLMs, slowly
Pi 5 8GB
4× A76 @ 2.4 GHz
~17 GB/s
8 GB
—
~3× Pi 4-class LLM throughput
Pi 5 16GB
4× A76 @ 2.4 GHz
~17 GB/s
16 GB
—
Only Pi that fits a 30B-A3B MoE; $305 list (Raspberry Pi)
The headline number for LLM work is memory bandwidth. Llama 3.2 3B at Q4_K_M is a 2.0 GB file that must stream through the memory bus for every token. That caps a Pi 4-class board near 2 tok/s (the Pi 400 measures 1.60) and a Pi 5 near 8 tok/s (it measures 4.61). Geerling's ai-benchmarks links the run behind each Pi row; his Pi 5 FAQ post covers the board's power and efficiency.
What language models actually run on a Pi 4 8GB?
RAM figures are approximate Q4_K_M footprints. Public Pi 4-class tok/s measurements are sparse, so only Geerling's Pi 400 row is filled in; the rest are left blank rather than estimated.
Anything 3B and under at q4 fits in 8 GB, but the realistic Pi 4 sweet spot in 2026 is the sub-1.5B tier. For 3B models, the 1.60 tok/s Pi 400 run is the ceiling, not the floor; 7B+ loads but crawls.
Short prompts also matter: on a CPU-only Pi a long context can take longer to ingest than the reply takes to generate, so for agentic or RAG workloads prefill is the real bottleneck.
What runs on a Pi 5 (8GB and 16GB)
The Pi 5's extra memory bandwidth is the whole story here. On the same Llama 3.2 3B run, Geerling's ai-benchmarks records 4.61 tok/s on a Pi 5 8GB against 1.60 on a Pi 400, close to 3×. Rows are from that dataset (Ollama, CPU only) unless marked otherwise.
Below the chat models sits a function-calling tier: the 14 MB Needle model decodes at 248–314 tok/s on a Pi 5 8GB CPU, per Raspberry Pi. Sub-1B chat on a Pi 5 is now responsive, and 3B-class chat works if you can wait a few seconds per sentence. The 16GB board adds a 30B MoE that runs faster than a 3B dense model and, per Geerling's AI HAT+ 2 write-up, completed a TODO-app task the tiny models did not complete well. The board draws about 10.5W during that run (2.7–3W idle), per issue #47. Runtime choice now matters too: on the same Pi 5 8GB, Google's LiteRT-LM ran Gemma 4 E2B about 29% faster than llama.cpp (9 vs 7 tok/s) at roughly a third of the peak memory, per Raspberry Pi. Clustering is the other route to bigger models: four 16GB boards reached 15.143 tok/s on Qwen3 30B-A3B in a distributed-llama study, which is not yet independently reproduced and costs four $305 boards at current list price.
It is a useful on-device assistant for home-automation NLU, simple code tasks or summarizing, not a serious LLM host.
The biggest practical upgrade if you have a Pi 5 is heat. The board holds 2.4 GHz only when thermal headroom allows. The official Active Cooler or a small case fan keeps sustained inference workloads from throttling.
Where the Pi actually shines: vision and audio
Language models stress the Pi's weakest part, memory bandwidth. Vision and audio models stress compute, which the Pi is surprisingly good at. Workloads that run in real time or close to it on a Pi 5:
YOLO object detection. On a Hailo accelerator, Geerling measured a YOLO model at 30 fps from a Camera Module 3, about 10× faster than on the Pi's CPU. The earlier AI HAT+ and the $70 AI Camera handle the same job; Hailo's docs live in the Hailo Developer Zone.
Wake-word detection (openWakeWord, Picovoice): low-millisecond latency on a single core.
Whisper.cpp tiny/base transcription: real-time-class on a Pi 5, slower on a Pi 4. Bigger Whisper models fall behind real time.
Background removal / segmentation: usable at low frame rates for smart cameras.
TTS (Piper): faster than real time on a Pi 5.
The difference is weight reuse: a small YOLO model reuses the same weights across thousands of frames, while an LLM walks every byte of its weights for every token.
The AI HAT+ 2, the AI HAT+ and other accelerators
For vision, the Pi 5 plus a Hailo-based AI HAT+ is still the configuration to buy.Raspberry Pi launched the original AI Kit in mid-2024: a Hailo-8L delivering 13 TOPS over PCIe Gen 2 x1. The current AI HAT+ comes in 13 TOPS (Hailo-8L) and 26 TOPS (Hailo-8) versions.
The AI HAT+ 2 is the first Pi accelerator that runs LLMs. Per Raspberry Pi's launch post, launch models were DeepSeek-R1-Distill 1.5B, Llama 3.2 1B, Qwen2.5-Coder 1.5B, Qwen2.5-Instruct 1.5B and Qwen2 1.5B. The same post says its vision performance is "broadly equivalent" to the 26 TOPS AI HAT+. Raspberry Pi's documentation puts the ceiling at LLMs and VLMs up to ~6B parameters.
Measured, the CPU wins. In CNX Software's review, the Hailo ran on a Pi 5 2GB and the CPU figures came from a CM5, which uses the same BCM2712 SoC:
Model
AI HAT+ 2 (Hailo-10H) tok/s
CM5 CPU (Ollama) tok/s
Qwen2.5 1.5B Instruct
6.74
11.73
Qwen2.5 Coder 1.5B
8.06
10.26
DeepSeek R1 Distill 1.5B
6.72
9.04
Llama 3.2 3B
2.60
4.78
CNX cautions that the Hailo figures may include prompt-evaluation time, so the true decode rate could be slightly higher. The Hailo does win on power: CNX measured 7.2–7.6W at the wall for the Pi 5 plus HAT against 10.2–10.6W for the CM5 CPU run.
Geerling reached the same conclusion ("The Pi's built-in CPU trounces the Hailo 10H"), blaming the Hailo's 3W power limit versus ~10W for the SoC, and recommends a 16GB Pi 5 over the $130 HAT for LLMs. Buy the AI HAT+ 2 for power-constrained vision-plus-small-LLM products or as a Hailo-10H dev kit, not as a chat accelerator.
Alternatives worth knowing:
Sixfab AI HAT+ (DEEPX DX-M1M): 25 TOPS at about 3 W with 2 GB on-module RAM; it ran Qwen3 1.7B at 4.64 tok/s on a Pi 5 8GB and MobileNetV2 at 2,361 fps, per Raspberry Pi's October 2026 post. Like the Hailo-10H, it is a vision-first part, not a faster chat engine.
Coral USB Accelerator (Edge TPU): cheaper, lower TOPS; only if you already run the Coral stack.
Jetson Orin Nano Super 8GB: the natural step-up — 67 sparse TOPS, 102 GB/s (about 6× the Pi 5), $249, and 19.14 tok/s on Llama 3.1 8B INT4 per NVIDIA.
Storage matters more than people think
SD-card I/O is slow and inconsistent for loading multi-gigabyte GGUFs. Use a USB 3.0 SSD on a Pi 4 or an NVMe HAT on a Pi 5. Even an entry-grade SATA SSD like the Crucial BX500 1TB or SanDisk Ultra 3D 1TB in a USB 3.0 enclosure cuts load times sharply.
The Pi 5's M.2 HAT+ adds an NVMe slot at PCIe Gen 2 x1 (theoretical 500 MB/s). That matters more than ever now that the useful 16GB-board models are 10 GB files.
Common pitfalls
No active cooling. A Pi 4 throttles quickly under sustained inference; use a heatsink and fan, or the Active Cooler on a Pi 5.
Running off SD card. Long pauses mid-response and hangs under load. Switch to a USB SSD or NVMe.
Wrong quant. q8/fp16 read ~2× the bytes per token, so they run about half as fast with little benefit at 1B–3B. Stay at Q4_K_M; for MoE models on 16GB, tuned sub-3-bit quants are what make them fit.
Buying the AI HAT+ 2 for chat speed. Per the reviews above it is slower than the CPU on every tested LLM. Buy the 16GB Pi 5 instead.
Underpowered PSU. The Pi 5 wants the official 27W PSU. A 15W generic supply browns out under sustained accelerator + CPU load and can corrupt SD cards.
Expecting a Pi 4 to hit Pi 5 numbers. Many "Pi can run Llama!" videos silently use a Pi 5. Check before buying: a Pi 4 8GB in 2026 is cheaper, but it is meaningfully slower.
When NOT to use a Pi for AI
A Pi is the wrong tool when:
You need a 7B+ dense language model at conversational speed. Buy a Jetson Orin Nano Super or an Apple Silicon Mac mini.
You need batched inference. In practice the Pi serves one request at a time.
You need GPU-accelerated training or fine-tuning. The Pi has no usable GPU compute for ML.
You need more than ~10 tok/s on anything bigger than a ~1.5B dense model. Bandwidth is the wall (see the ceilings above).
A realistic 2026 Pi-based AI rig
The build people actually keep:
Raspberry Pi 5 16GB (or 8GB if you'll stay at ≤3B models) + Active Cooler + 27W PSU. Budget for 2026 pricing: $305 for the 16GB per Raspberry Pi and $175 for the 8GB per a forum price summary.
AI HAT+ (13 or 26 TOPS) for vision pipelines. The AI HAT+ 2 only if you need on-HAT LLMs under a tight power budget.
NVMe via the M.2 HAT+, or a 1 TB SATA SSD over USB 3.0 (Crucial BX500 or SanDisk Ultra 3D), for model storage
Ollama or llama.cpp as the LLM serving layer: Qwen3 0.6B or Gemma 3 1B for fast replies, Gemma 4 E2B (via LiteRT-LM) for a newer small model, Llama 3.2 3B for more reliable answers, and on 16GB a 30B-A3B MoE quant for harder tasks
Frigate or Viseron as the vision orchestrator on top of the Hailo accelerator
Optional: a USB microphone array + Piper TTS for a voice-loop assistant
This runs a small chat assistant, a vision pipeline and Whisper transcription on one board (keep it cooled). It is genuinely useful for a single user and will never compete with a discrete-GPU machine, and that's fine.
As an Amazon Associate, SpecPicks earns from qualifying purchases; we also earn on qualifying eBay purchases via the eBay Partner Network. Prices shown were last tracked at crawl time and may vary — check the listing for the current price.
Yes, but only small ones, and slowly. Per Jeff Geerling's ai-benchmarks, a Pi 4-class board (the Pi 400) generates Llama 3.2 3B at 1.60 tokens per second, which is close to its memory-bandwidth ceiling. The realistic sweet spot on a Pi 4 8GB is the sub-1.5B tier, such as Qwen3 0.6B, Gemma 3 1B and Qwen 2.5 1.5B at 4-bit quantization. Models of 7B and above load but are far too slow for interactive use.
How fast is a Raspberry Pi 5 for local LLMs?
As of October 2026, a Pi 5 running CPU-only under Ollama generates about 16.6 tokens per second on Qwen3 0.6B and 10 tokens per second on Gemma 3 1B, per Adafruit's published runs, and 4.61 tokens per second on Llama 3.2 3B, per Jeff Geerling's ai-benchmarks. That is roughly 3 times a Pi 400 on the same 3B model. Google's newer Gemma 4 E2B decodes at 9 tokens per second on a Pi 5 8GB under LiteRT-LM and 7 under llama.cpp, per Raspberry Pi's LiteRT and Gemma write-up. The limit is the Pi 5's roughly 17 GB/s memory bandwidth, not its CPU clock.
Can a Raspberry Pi 5 run a 30B model?
Only the 16GB Pi 5, which now lists at $305 per Raspberry Pi's product page, and only a mixture-of-experts model. Jeff Geerling measured 7.99 tokens per second on Qwen3 30B-A3B Instruct 2507 using ByteShape's 2.70-bit GGUF, a 10.3 GB file that barely fits with context. It runs faster than a 3B dense model because only about 3 billion of its 30.5 billion parameters are read per token. It will not fit on an 8GB Pi 5.
Is the Raspberry Pi AI HAT+ 2 worth buying for LLMs?
Not for speed. In CNX Software's review the Hailo-10H on the $130 AI HAT+ 2 ran Qwen2.5 1.5B at 6.74 tokens per second versus 11.73 on the same SoC's CPU, and Llama 3.2 3B at 2.60 versus 4.78. Jeff Geerling also found the Pi's CPU faster and recommends a 16GB Pi 5 instead, although that board now lists at $305 per Raspberry Pi's product page. The HAT does use less power, about 7.2 to 7.6 watts versus 10.2 to 10.6 watts in CNX's test, and it is strong for vision, where Geerling saw YOLO run about 10 times faster than on the CPU.
Should I use an SD card or SSD for AI workloads on a Pi?
Use an SSD. SD cards are slow and inconsistent at loading multi-gigabyte model files and can cause pauses and hangs under sustained load. A SATA SSD in a USB 3.0 enclosure on a Pi 4, or an NVMe drive on the Pi 5's M.2 HAT+ (PCIe Gen 2 x1, about 500 MB/s theoretical), loads models much faster, which matters now that the useful 16GB-board models are 10 GB files.
Is a Jetson Orin Nano faster than a Raspberry Pi 5 for local LLMs?
Yes, by roughly 4 to 5 times on the same model. Llama 3.2 3B decodes at 5.71 tokens per second on a Pi 5 16GB under llama.cpp, per the pi5-llm-arena benchmark, and at 25.24 tokens per second on a Jetson Orin Nano 8GB in its 25 W mode, per Turing Pi's llama-bench results. Qwen3 1.7B runs at 6.89 tokens per second on a Pi 5 8GB under Ollama versus 34.16 on the Jetson. NVIDIA lists the Orin Nano Super at 102 GB/s of memory bandwidth and $249, while the Pi 5 16GB lists at $305 per Raspberry Pi. The Pi 5 16GB's advantage is capacity: it fits a 30B mixture-of-experts quant that the 8GB Jetson cannot.
