Skip to main content
24 GB VRAM 27 models checked 21 guides

RTX 3090 for Local LLMs: What It Runs, What It Costs, Every Guide

The RTX 3090's 24 GB holds models up to about DeepSeek-R1-Distill-Qwen 32B entirely in VRAM: its 19.9 GB Q4_K_M file leaves 4.1 GB for context (per the SpecPicks Will-It-Run matrix). New, it is $1,915 this week, $79.79 per GB of VRAM, per the Amazon listings SpecPicks tracks (price may vary).

By SpecPicks Editorial — AI Rigs Team

What fits in 24 GB

Each row is the model’s real GGUF file size at the quantization shown (Q4_K_M unless noted) against the card’s 24 GB, with the median of published, source-linked generation-speed runs that pass a memory-bandwidth sanity check. Open a row for the sources behind it.

ModelFileVerdictMedian speed
Llama 3.2 1B 0.8 GB Q4_K_M Runs 267 tok/s 1 cited run
Llama 3.2 3B 2 GB Q4_K_M Runs no cited run
Mistral 7B 4.4 GB Q4_K_M Runs no cited run
Qwen2.5 7B 4.7 GB Q4_K_M Runs no cited run
Llama 3.1 8B 4.9 GB Q4_K_M Runs 92 tok/s 3 cited runs
DeepSeek-R1-Distill-Llama 8B 4.9 GB Q4_K_M Runs no cited run
Qwen3 8B 5 GB Q4_K_M Runs 101 tok/s 2 cited runs
Gemma 2 9B 5.8 GB Q4_K_M Runs no cited run
Gemma 3 12B 7.3 GB Q4_K_M Runs no cited run
Qwen2.5 14B 9 GB Q4_K_M Runs 55.8 tok/s 3 cited runs
DeepSeek-R1-Distill-Qwen 14B 9 GB Q4_K_M Runs no cited run
Qwen3 14B 9 GB Q4_K_M Runs 61.1 tok/s 2 cited runs
Phi-4 14B 9.1 GB Q4_K_M Runs no cited run
gpt-oss-20b 12.1 GB MXFP4 Runs 148 tok/s 1 cited run
Gemma 3 27B 16.6 GB Q4_K_M Runs no cited run
Gemma 2 27B 16.7 GB Q4_K_M Runs no cited run
Qwen3 30B-A3B 18.6 GB Q4_K_M Runs 154 tok/s 1 cited run
Qwen3 32B 19.8 GB Q4_K_M Runs 32.7 tok/s 2 cited runs
Qwen2.5 32B 19.9 GB Q4_K_M Runs 23 tok/s 2 cited runs
DeepSeek-R1-Distill-Qwen 32B 19.9 GB Q4_K_M Runs no cited run
Qwen3 32B (Q6_K) 26.9 GB Q6_K Offload no cited run
Qwen2.5 32B (Q6_K) 26.9 GB Q6_K Offload no cited run
DeepSeek-R1-Distill-Qwen 32B (Q6_K) 26.9 GB Q6_K Offload no cited run
Gemma 3 27B (Q8_0) 28.7 GB Q8_0 Offload no cited run
Llama 3.1 70B 42.5 GB Q4_K_M Offload no cited run
Llama 3.3 70B 42.5 GB Q4_K_M Offload no cited run
DeepSeek-R1-Distill-Llama 70B 42.5 GB Q4_K_M Offload no cited run

What it costs this week

  • New (Amazon): $1,915 · $79.79 per GB of VRAM
  • Used (eBay median, 7 days): not tracked — the eBay feed SpecPicks reads has no used listings on file for this card, so there is no used median; check live eBay listings for current used prices

View Current Price →DetailsUsed on eBay

*Tracked snapshot read 2026-11-13, not a live quote: price may vary. Its row in this week’s LLM GPU Price Index →

Every RTX 3090 guide, by question

21 SpecPicks guides cover this card. They are grouped by the question they answer, most-read first.

Compared with other cards and machines 9 guides

Backends, drivers and runtimes 3 guides

Specific models 6 guides

Builds, buying and value 1 guide

Everything else 2 guides

Where to go next

As an Amazon Associate, SpecPicks earns from qualifying purchases. As an eBay Partner Network member, SpecPicks may earn a commission on eBay links.

This page is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.