The local-LLM GPU question has one answer per memory size, and VRAM, not compute, decides it. Per Hardware Corner's local-LLM GPU ranking, an RTX 5060 Ti 16GB generates 51.41 tok/s on Qwen3 8B at a 16K context, and a 24GB RTX 3090 generates 52.14 tok/s on Qwen3 14B at 16K against 32.91 tok/s for the 5060 Ti. The community threads SpecPicks has counted settle on five picks: an RTX 3060 12GB for 9-14B dense models and 35B mixture-of-experts (MoE) models with offload, an RTX 5060 Ti 16GB for a new card, a used RTX 3090 for 24 GB, a 48 GB card for 70B models at 4-bit, and a 128 GB Strix Halo mini PC for 120B-class MoE models. Each tier below names what fits, one cited speed, the pick, and the page that counts the threads behind it.
The tier picks at a glance
Each row has its Amazon price when one was read in the last 24 hours, and its own buy button. The picks are alternatives, one per memory size, not parts of one build.
| Part | Pick | Why | Price | Buy |
|---|---|---|---|---|
| GPU (8 GB) | GIGABYTE GeForce RTX 5060 WINDFORCE OC 8G | 7-8B models at 4-bit with a short context | $459.99 (2026-10-21 PT) | View Current Price |
| GPU (12 GB) | MSI GeForce RTX 3060 12GB Ventus 2X | Dense 9-14B at 4-bit; 35B-A3B MoE with expert offload | $499.99 (2026-10-21 PT) | View Current Price |
| GPU (16 GB) | ASUS Dual GeForce RTX 5060 Ti 16GB | Dense 14B with long context; 20-30B MoE | $765.99 (2026-10-21 PT) | View Current Price |
| GPU (48 GB) | MAXSUN Intel Arc Pro B60 Dual 48G Turbo | 70B at Q4_K_M on one board | $1,939.00 (2026-10-21 PT) | View Current Price |
| System (128 GB unified) | GMKtec EVO-X2 Ryzen AI Max+ 395 128GB | 120B-class MoE models such as gpt-oss-120b | $3,649.99 (2026-10-21 PT) | View Current Price |
*Price sourced from Amazon.com. Last updated 2026-10-21 PT. Price and availability subject to change.
Amazon prices are read from Amazon.com listings within the last 24 hours and change often; a pick with no current price shows no price. The 24 GB pick is a used RTX 3090, which the threads buy on eBay rather than new, so it is covered in its own section below.
8 GB: what fits, and why the threads treat it as the floor
8 GB holds a 7-8B model at 4-bit with room for a short context. Larger dense models spill into system RAM and slow down sharply. The RTX 3060 12GB consensus page and the RTX 3060 12GB vs 3060 Ti 8GB comparison cover the trade: the 8 GB card is often faster on paper, and the 12 GB card runs the models people actually ask about. Buy 8 GB for local LLMs only if the card is already in your PC.
12 GB: the RTX 3060 consensus
The RTX 3060 12GB Reddit consensus counts 13 r/LocalLLaMA threads and finds two setups owners settle on:
- A 35B-A3B MoE model with expert layers in system RAM. One owner reported about 46.8 tok/s generation for Qwen3.6-35B-A3B IQ4_XS on a 3060 with 32 GB of DDR4 (r/LocalLLaMA thread).
- A dense 9-14B model at 4-bit, entirely in VRAM. Per Hardware Corner's RTX 3060 benchmarks, dense Qwen3 14B at Q4 generates 22.7 tok/s at a 16K context and 31.2 tok/s at 4K.
gpt-oss-20b also fits: an RTX 3060 owner measured 75 tok/s with the whole model and a 2,048-token context in VRAM, and 64 tok/s at a 16K context with two layers' experts on the CPU (llama.cpp discussion #15396). The full model-by-model fit tables are in the complete RTX 3060 12GB local LLM guide, and the gpt-oss threads are counted in gpt-oss on a 12GB GPU: community consensus.
16 GB: the new-card pick
For a new card, the 16GB local-LLM build consensus lands on an RTX 5060 Ti 16GB (or an RX 9060 XT 16GB if you accept ROCm or Vulkan setup) with 64 GB of system RAM for MoE offload. Per Hardware Corner, the 5060 Ti 16GB generates 51.41 tok/s on Qwen3 8B and 32.91 tok/s on Qwen3 14B at 16K. The extra 4 GB over the 3060 mostly buys context length and a larger quant at 14B, not a new model class; 27-32B dense models still need 24 GB.
24 GB: the used RTX 3090
The used RTX 3090 consensus counts nine threads. The 3090 is still the 24 GB card most local-LLM builds start from, with 24 GB on one card (NVIDIA spec page) and 52.14 tok/s on Qwen3 14B at 16K per Hardware Corner. It is no longer cheap. Per GPU Poet's eBay tracker, the daily average of the three lowest RTX 3090 listings ran $1,242 to $1,395 in September 2026. The threads' verdict: buy the 3090 if your models need more than 16 GB (dense 27-32B at 4-bit), and otherwise buy a new 16 GB card. Current listings and the card's benchmarks are on the RTX 3090 benchmark page. Two 3060s are the cheaper route to 24 GB, compared in two 3060s or one used 3090 for a 32B model.
48 GB: 70B models at 4-bit
Llama 3.3 70B at Q4_K_M is a 42.5 GB file, so 48 GB is the first tier that holds it on GPU. The Arc Pro B60 Dual vs dual RTX 3060 comparison picks Maxsun's Arc Pro B60 Dual 48G for 70B models: two GPUs on one board, each with 456 GB/s of bandwidth (Maxsun). The trade is software. Intel's stack runs llama.cpp and IPEX-LLM, while every inference tool supports NVIDIA's CUDA first. The CUDA route to 48 GB is a workstation card such as the RTX A6000, bought used.
128 GB unified: Strix Halo for 120B-class MoE
For models larger than any discrete card holds, the pick is a Ryzen AI Max+ 395 ("Strix Halo") mini PC with 128 GB of unified memory. Per the community Strix Halo Guide benchmark log, a 128 GB system runs gpt-oss-120b at 56.14 tok/s on llama.cpp and Qwen3.6-35B-A3B at about 62.6 tok/s in the Vulkan backend. Dense models are its weak spot: it decodes Llama 3.1 70B at Q4_K_M at 4.97 tok/s, about a third of an M3 Ultra Mac Studio's 14.08 tok/s on the same test, per Jeff Geerling's ai-benchmarks results. The 128 GB Framework Desktop listed at $3,449 on October 16, 2026. The benchmarks by model and quant are in Strix Halo LLM benchmarks, and the box is set against two 3060s in Ryzen AI Max+ 395 128GB vs dual RTX 3060.
How to pick your tier
- Name the largest model you want to run, at the quant you will use. A 4-bit file is a little over half the parameter count in GB (Llama 3.3 70B at Q4_K_M is 42.5 GB). Add room for the context.
- If it fits in VRAM, buy the cheapest card with that much. Generation speed tracks memory bandwidth once the model fits, which is why a 3090 outruns a newer 16 GB card at 14B in the figures above.
- If it is a MoE model, system RAM counts too. Expert offload is how a 12 GB card runs a 35B-A3B model at the speed reported above. The threads pair 12-16 GB cards with 32-64 GB of RAM for this.
- If it is dense and over 48 GB, the choice is a multi-GPU rig or unified memory. Unified memory holds more but decodes dense models slowly, per the Strix Halo figures above.
Every page this hub summarizes
- RTX 3060 12GB local LLM: Reddit consensus on what runs
- gpt-oss on a 12GB GPU: community consensus
- Local-LLM PC build on a 16GB card: the community consensus
- Used RTX 3090 vs a new GPU for local LLMs: Reddit consensus
- Arc Pro B60 Dual 48GB vs dual RTX 3060 12GB for 70B models
- Strix Halo LLM benchmarks: cited tok/s by model and quant
- The complete RTX 3060 12GB local LLM guide
Citations and sources
- Hardware Corner — GPU ranking for local LLMs
- Hardware Corner — RTX 3060 12GB LLM benchmarks
- r/LocalLLaMA — Qwen 35B-A3B is very usable with 12GB of VRAM
- llama.cpp discussion #15396 — RTX 3060 gpt-oss-20b results
- GPU Poet — RTX 3090 eBay prices, September 2026
- NVIDIA — GeForce RTX 3090 specifications
- Maxsun — Intel Arc Pro B60 Dual 48G Turbo
- Strix Halo Guide — benchmark log
- Jeff Geerling — ai-benchmarks
- Framework — Framework Desktop (Ryzen AI Max 300)
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
