Run Qwen3 Locally — Hardware Tiers, Tok/s & Build Guide
Alibaba's flagship open-weight model family with strong code, math, and tool-use performance. Available variants: 0.6B1.7B4B8B14B32B30B-A3B MoE235B-A22B MoE.
Qwen3 currently leads the open-weight leaderboard on tool-use and code benchmarks at every parameter tier. The 32B variant is the sweet-spot model for a single 24-32GB GPU local rig — runs cleanly at Q4_K_M on an RTX 4090 or RTX 5090. Cited 5090 runs reach ~58–61 tok/s at short context and 43.8 tok/s at 32K context (Q4_K_XL, Hardware Corner).
Hardware tiers for running Qwen3
🧪
Minimum (8B/14B at Q4)
Perfect for chatbot + coding use cases on smaller variants. 16GB lets you run 14B at Q4 with comfortable context.
VRAM12 GB
System RAM32 GB DDR5
Throughput~26 tok/s on 14B Q4 (RTX 4060 Ti 16 GB, median of 12 cited runs) · ~29 tok/s on the same 14B Q4 band (RTX 3060 12 GB, median of 26 runs; its 360 GB/s bus beats the 4060 Ti's 288 GB/s, but 12 GB leaves little room for context at 14B, so the 16 GB card is the pick)
*Price sourced from Amazon.com. Price and availability subject to change.
🏆
Recommended (32B at Q4)
Sweet spot for daily-driver Qwen3 use. 24 GB fits 32B at Q4_K_M with 32–64K context comfortably; 32 GB (RTX 5090) extends context to full 128K and keeps latency low.
VRAM24 GB
System RAM64 GB DDR5
Throughput~34–38 tok/s on Qwen3 32B Q4_K_M (RTX 4090, 2 cited runs: awesomeagents.ai, LocalScore) · ~58–61 tok/s at the same Q4_K_M (RTX 5090, 2 cited runs: DatabaseMart, Hardware Corner)
*Price sourced from Amazon.com. Price and availability subject to change.
⚡
Luxury (235B-A22B MoE at Q4, expert offload)
Qwen3 235B-A22B is roughly 140 GB of weights at Q4, so no single consumer card holds it. With llama.cpp expert offload, 48 GB of VRAM keeps attention and the shared layers on the GPU while the experts sit in system RAM; only 22B parameters are active per token. RTX PRO 6000 Blackwell (96 GB) moves more of the model onto the card.
VRAM48 GB
System RAM128–192 GB DDR5
ThroughputBound by system-RAM bandwidth for the offloaded experts — no cited single-rig figure yet
A 24 GB card (RTX 4090, RTX 3090, or RTX 5090) at Q4_K_M quantization. With 16 GB cards you can fit 14B comfortably but the 32B variant requires either Q3 quant (quality drop) or CPU offload (latency drop).
Q4_K_M vs FP16 — what's the quality cost?
On benchmarks, Q4_K_M loses about 1-2% on average vs FP16 — well below practical-use noticeable. Q3_K_M starts to show: 4-6% drop and noticeable on edge cases. Always pick Q4_K_M as the default if VRAM permits.
Can I run Qwen3 on a Mac?
Yes, with caveats. A Mac Studio M3 Ultra with 192 GB unified memory holds even Qwen3 235B-A22B at Q4 (about 140 GB of weights) — silent, low-power, and the capacity is enormous. An M4 Max with 128 GB runs up to 32B, or the 30B-A3B MoE, comfortably.
Is Qwen3 better than Llama 4 / DeepSeek V3?
Depends on your workload. Qwen3 leads on tool-use and coding benchmarks; Llama 4 has better creative writing; DeepSeek V3 has the best math and reasoning at very large sizes. For a local-rig daily driver, Qwen3 32B is the most balanced.
More guides & deep dives from the SpecPicks archive
