Skip to main content
32 GB VRAM 27 models checked 38 guides

RTX 5090 for Local LLMs: What It Runs, Every Guide

The RTX 5090's 32 GB holds models up to about Gemma 3 27B entirely in VRAM: its 28.7 GB Q8_0 file leaves 3.3 GB for context (per the SpecPicks Will-It-Run matrix).

What fits in 32 GB

Each row is the model’s real GGUF file size at the quantization shown (Q4_K_M unless noted) against the card’s 32 GB, with the median of published, source-linked generation-speed runs that pass a memory-bandwidth sanity check. Open a row for the sources behind it.

ModelFileVerdictMedian speed
Llama 3.2 1B 0.8 GB Q4_K_M Runs no cited run
Llama 3.2 3B 2 GB Q4_K_M Runs no cited run
Mistral 7B 4.4 GB Q4_K_M Runs no cited run
Qwen2.5 7B 4.7 GB Q4_K_M Runs no cited run
Llama 3.1 8B 4.9 GB Q4_K_M Runs 180 tok/s 2 cited runs
DeepSeek-R1-Distill-Llama 8B 4.9 GB Q4_K_M Runs no cited run
Qwen3 8B 5 GB Q4_K_M Runs 186 tok/s 1 cited run
Gemma 2 9B 5.8 GB Q4_K_M Runs no cited run
Gemma 3 12B 7.3 GB Q4_K_M Runs no cited run
Qwen2.5 14B 9 GB Q4_K_M Runs 89.9 tok/s 1 cited run
DeepSeek-R1-Distill-Qwen 14B 9 GB Q4_K_M Runs 89.1 tok/s 1 cited run
Qwen3 14B 9 GB Q4_K_M Runs 124 tok/s 1 cited run
Phi-4 14B 9.1 GB Q4_K_M Runs 130 tok/s 1 cited run
gpt-oss-20b 12.1 GB MXFP4 Runs 298 tok/s 1 cited run
Gemma 3 27B 16.6 GB Q4_K_M Runs no cited run
Gemma 2 27B 16.7 GB Q4_K_M Runs no cited run
Qwen3 30B-A3B 18.6 GB Q4_K_M Runs 234 tok/s 1 cited run
Qwen3 32B 19.8 GB Q4_K_M Runs 58 tok/s 3 cited runs
Qwen2.5 32B 19.9 GB Q4_K_M Runs 54 tok/s 2 cited runs
DeepSeek-R1-Distill-Qwen 32B 19.9 GB Q4_K_M Runs 54 tok/s 3 cited runs
Qwen3 32B (Q6_K) 26.9 GB Q6_K Runs no cited run
Qwen2.5 32B (Q6_K) 26.9 GB Q6_K Runs no cited run
DeepSeek-R1-Distill-Qwen 32B (Q6_K) 26.9 GB Q6_K Runs no cited run
Gemma 3 27B (Q8_0) 28.7 GB Q8_0 Runs no cited run
Llama 3.1 70B 42.5 GB Q4_K_M Offload no cited run
Llama 3.3 70B 42.5 GB Q4_K_M Offload no cited run
DeepSeek-R1-Distill-Llama 70B 42.5 GB Q4_K_M Offload no cited run

What it costs this week

  • New (Amazon): no quotable listing this week (none priced within 1.6× of launch MSRP in the last 24 hours)
  • Used (eBay median, 7 days): not tracked — the eBay feed SpecPicks reads has no used listings on file for this card, so there is no used median; check live eBay listings for current used prices

View Current Price →DetailsUsed on eBay

*Tracked snapshot, not a live quote: price may vary. Its row in this week’s LLM GPU Price Index →

Every RTX 5090 guide, by question

38 SpecPicks guides cover this card. They are grouped by the question they answer, most-read first.

Compared with other cards and machines 20 guides

8 more

Backends, drivers and runtimes 2 guides

Front-ends, agents and tools 2 guides

Specific models 5 guides

Builds, buying and value 2 guides

Everything else 7 guides

Where to go next

As an Amazon Associate, SpecPicks earns from qualifying purchases. As an eBay Partner Network member, SpecPicks may earn a commission on eBay links.

This page is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.