DeepSeek's 671B mixture-of-experts flagship — strong math and reasoning, with a memory footprint far beyond any single consumer GPU. Available variants: 671B MoE (active ~37B).
DeepSeek V3.2 is a 671B-parameter mixture-of-experts model that routes each token through about 37B active parameters, so it generates faster than its size suggests — but every expert still has to sit in memory. At Q4 the weights alone are roughly 370–400 GB, over ten times the VRAM of an RTX 5090. Running it locally means a 24–32 GB card with the experts offloaded to hundreds of GB of system RAM, or a 512 GB unified-memory Mac. If you want strong reasoning on one consumer GPU, a 32B-class dense model is the realistic choice.
Hardware tiers for running DeepSeek V3.2
🧪
Minimum (671B at ~2-bit, GPU + expert offload)
A ~2-bit dynamic quant shrinks the weights to roughly 170–230 GB. The 24 GB card holds attention and the shared expert; the routed experts live in system RAM (llama.cpp tensor overrides or ktransformers). Workable for batch jobs, slow for chat, and ~2-bit loses noticeably more quality than Q4.
VRAM24 GB
System RAM256 GB system RAM
ThroughputLow single-digit tok/s — bound by system-RAM bandwidth, not the GPU
*Price sourced from Amazon.com. Price and availability subject to change.
🏆
Recommended (671B at Q4, GPU + server-class RAM)
Q4 weights (~370–400 GB) fit in 512 GB of RAM with room for context. Throughput scales with memory channels, so an EPYC or Xeon platform matters more than the GPU tier — the card mainly accelerates attention and prompt processing.
VRAM24 GB
System RAM512 GB DDR5 on an 8–12-channel server or workstation platform
ThroughputRoughly 10–14 tok/s reported with ktransformers on a dual-socket Xeon plus one 24 GB card
*Price sourced from Amazon.com. Price and availability subject to change.
⚡
Luxury (671B at Q4, fully in fast memory)
Keeping all ~400 GB of Q4 weights in fast memory takes a 512 GB Mac Studio M3 Ultra or four to eight 96 GB RTX PRO 6000 Blackwell cards. This is datacenter money, not a consumer build.
VRAM96 GB
System RAM512 GB unified memory, or 4–8 × 96 GB GPUs
Throughput~20 tok/s at 4-bit on a Mac Studio M3 Ultra 512 GB (MLX)
Can DeepSeek V3.2 run on a single RTX 4090 or RTX 5090?
Not on its own. DeepSeek V3.2 is 671B parameters in total, and at Q4 the weights alone are about 370–400 GB — over ten times a 32 GB card. A 24–32 GB card can host the attention layers and shared expert while the routed experts are offloaded to system RAM (llama.cpp tensor overrides or ktransformers). That needs 256–512 GB of RAM and runs at single- to low-double-digit tok/s.
Why is a 671B MoE model faster than its size suggests?
Mixture-of-experts. Each token passes through only about 37B active parameters, so compute per token is closer to a ~37B dense model than a 671B one. Memory is the catch: all 671B parameters must be loaded, because different tokens route to different experts.
Is there a small DeepSeek V3.2 that fits a 16 or 24 GB card?
No. DeepSeek V3.x ships only as the 671B MoE. The 16B (2.4B active) and 236B (21B active) sizes some guides list belong to the older DeepSeek-V2-Lite and DeepSeek-V2. For a single 24 GB card, a 32B-class dense model at Q4 is the realistic alternative.
DeepSeek vs Qwen3 for coding?
Qwen3 wins on real-world coding tasks (better tool-use, more idiomatic generations). DeepSeek wins when the task is more mathematical (algorithm design, complexity analysis). For full-stack dev: Qwen3. For algo / interview prep: DeepSeek.
More guides & deep dives from the SpecPicks archive
