Meta's flagship open-weight family — strong creative writing and instruction-following. Available variants: 8B 17B 70B 405B.
As an Amazon Associate, SpecPicks earns from qualifying purchases. See our review methodology.
Llama 4 is the broadest-deployed open-weight family — every inference framework supports it day-one, and the 70B variant is the canonical "is this open model good enough for production" benchmark. The 17B mid-tier added in 4.1 makes single-GPU local hosting much more accessible than Llama 3.
8B Q4 fits in 8 GB but you want 12-16 GB for context room. 8B is a good lightweight assistant; coding and complex reasoning are limited.
*Price sourced from Amazon.com. Price and availability subject to change.
RTX 5090 (32 GB) lets you run the new 17B variant fully and 70B with light CPU offload. RTX 4090 (24 GB) handles 17B comfortably; 70B needs more aggressive offload and slows to ~8 tok/s.
*Price sourced from Amazon.com. Price and availability subject to change.
405B is too big for any consumer single-GPU rig. Mac Studio M3 Ultra (192 GB unified) is the only consumer machine that holds it in memory. Dual RTX 3090 NVLink runs 70B comfortably at 24+ tok/s.
*Price sourced from Amazon.com. Price and availability subject to change.
Llama 4 added a 17B mid-tier (Llama 3 had 8B → 70B with nothing in between), expanded context to 256K, and added native vision support across all sizes. Inference performance per parameter is similar to 3.3.
Only on Mac Studio M3 Ultra 192GB (or 96GB with aggressive offload). No consumer NVIDIA rig fits it natively. Dual RTX 3090 + 256 GB system RAM works at ~3 tok/s with heavy CPU offload — usable for batch jobs, not chat.
Used RTX 3090 ($800-1100) + 64GB DDR5 + Ryzen 7 7700X. Runs 17B Q4 at ~50 tok/s and 70B Q4 with offload at ~10 tok/s. Total build: $1500-1800.