As an Amazon Associate, SpecPicks earns from qualifying purchases.
On a 14B model at q4_K_M, LocalScore's accelerator database records 78.0 tok/s for the RTX 4090 against 55.8 tok/s for the RTX 3090 on its 3090 page — a 1.4× gap between two cards that hold exactly the same 24GB of VRAM. That equality is the whole point of this comparison. Unlike almost every other GPU upgrade in local inference, this one does not change which models you can run. It changes only how fast the identical model set runs, and how quickly a long prompt is ingested.
This synthesis puts the published runs SpecPicks tracks for both cards side by side, separates single-stream generation from prefill and from batched serving (three numbers that get quoted interchangeably and should not be), and states where the price difference is and is not defensible in 2026.
The short version
- Same 24GB ceiling. Both cards hold a 32B dense model at q4 with room for context. Both spill on 70B at q4.
- Generation: the 4090 leads by roughly 1.3–1.5× across the 8B, 14B and 32B classes in published runs.
- Prefill is the bigger gap. Hardware Corner records 9,121 tok/s prompt processing on the 4090 against 2,572.5 tok/s on the 3090 — a 3.5× difference that dominates long-context and agentic work.
- Sparse MoE: 195.8 tok/s on the 4090 for a 30B-class MoE at q4, against 135.7 tok/s for a 35B-A3B model on the 3090.
- Power: 450W against 350W, per SpecPicks' hardware records.
- Price: the lowest 4090 listing SpecPicks tracked on 2026-09-03 was $3,449.99, against $1,549.99 for a renewed 3090 Founders Edition.
Generation throughput, model by model
| Model class (q4) | RTX 3090 24GB | RTX 4090 24GB | Sources |
|---|---|---|---|
| Llama 3.1 / Qwen3 8B | 87.5–95.7 tok/s | 113–165 tok/s | LocalScore 3090, Hardware Corner 3090; llama.cpp GitHub, Hardware Corner 4090 |
| 14B dense | 52.1–55.8 tok/s | 78.0–82.8 tok/s | kunalganglani, LocalScore 3090; LocalScore 4090, Hardware Corner 4090 |
| Qwen3 / Qwen2.5 32B dense | 28.0–30.3 tok/s | 34.0–38.9 tok/s | Hardware Corner 3090, kunalganglani; LocalScore 4090, DatabaseMart |
| Sparse MoE (30–35B, ~3B active) | 135.7 tok/s | 195.8 tok/s | thc1006 on GitHub; Hardware Corner 4090 |
| 70B dense at q4 (offloaded) | 5.2 tok/s | 5.0–8.0 tok/s | GigaGPU; Mustafa.net |
Read the last row carefully: at 70B, the 4090's advantage nearly disappears, because the bottleneck is no longer the GPU. Both cards are moving most of the model across PCIe from system RAM, and PCIe bandwidth is the same on both.
The 8B row is the widest range in the table because it collects several runtimes. The 165 tok/s figure comes from a llama.cpp discussion run at Q4_K_M, while DatabaseMart's Ollama benchmark records 95.5 tok/s for the same card and model class under Ollama's defaults. That spread — same card, same model, 1.7× apart — is a runtime difference, not a hardware one, and it is larger than the gap between the two cards on that row.
Prefill: the number that actually separates them
Generation speed is what benchmarks quote; prompt processing is what you wait on when a coding agent hands the model 30,000 tokens of repository context before it says anything. Hardware Corner's 4090 table records 9,121 tok/s of prefill on Llama 3.1 8B at Q4_K_XL. Its 3090 table records 2,572.5 tok/s on Qwen3 8B at the same quantization class. LocalScore puts 3090 prefill at 3,536 tok/s on Llama 3.1 8B, and its 4090 page records 3,100 tok/s on a 14B model.
The consistent picture across those sources is a prefill advantage for the 4090 in the 2.5–3.5× range on comparable workloads — considerably larger than its 1.3–1.5× generation advantage. Ada's larger L2 cache and roughly 1.5× the memory bandwidth do more for the compute-bound prefill phase than for the bandwidth-bound generation phase.
If your workload is a chat window, this hardly matters. If it is an agent that re-ingests a large context every turn, it is the difference between the two cards.
Batched serving is a different number again
Spheron's benchmark write-up reports 2,550 tok/s for Llama 3.1 8B at FP16 on a 4090 under vLLM, and 650 tok/s for Qwen3 32B at AWQ. Those are aggregate throughput across concurrent requests, not single-stream generation, and they are not comparable to the llama.cpp figures above. The equivalent single-stream vLLM measurement on a 3090 is The Register's 42 tok/s on Llama 3.1 8B at FP16 using 16.1GB.
The distinction matters for one specific buyer: if you are serving several users or running parallel agent workers, batched throughput scales with compute and the 4090's advantage widens well past 1.5×. For a single user at a keyboard, it does not apply at all.
Synthetic scores and the rest of the build
PassMark records a G3D Mark of 38,066 for the 4090 against 26,548 for the 3090 — a 1.43× ratio that tracks the generation-speed gap in the table above more closely than the prefill gap does. TechPowerUp's 3DMark Time Spy figure puts the 4090 GPU score at 35,200.
SpecPicks' hardware records list a 450W board power for the 4090 and 350W for the 3090; every spec and benchmark row behind those numbers, with its source, is on the RTX 4090 benchmark page and the RTX 3090 benchmark page. Both are three-slot-class cards needing a 850W-plus supply in practice, so neither is the quiet-corner option — that is a tier further down, covered in RTX 3060 12GB vs RTX 3090 for local LLMs.
Price per token per second
Dividing the lowest tracked Amazon listing on 2026-09-03 by each card's 14B q4 figure from LocalScore:
| Tracked listing | 14B q4 tok/s | Dollars per tok/s | |
|---|---|---|---|
| RTX 3090 Founders Edition (renewed) | $1,549.99 | 55.8 | ~$27.78 |
| RTX 4090 Founders Edition | $3,449.99 | 78.0 | ~$44.23 |
Both cards are past end-of-production and priced by the used and grey market rather than by MSRP ($1,499 and $1,599 respectively at launch), so these ratios move. The direction has been stable, though: the 3090 is consistently the better throughput-per-dollar buy, and the 4090 is the better latency buy.
Which one should you buy
- You already own a 3090 and run 8–32B models in a chat window: the upgrade buys 1.3–1.5× on generation. That is real but rarely worth the difference in price on its own.
- You run coding agents or long-context work: the 4090, for the 2.5–3.5× prefill advantage. This is the strongest case for the newer card.
- You serve multiple concurrent users with vLLM: the 4090, where batched throughput scales with compute.
- You are building a first 24GB box on a budget: the 3090. It reaches the same model ceiling for roughly half the tracked price.
- You want two cards for 48GB: two 3090s cost less than one 4090 at current tracked listings and double the VRAM, which is the ceiling that actually determines what you can run. The trade is 700W of board power and an NVLink-free split across PCIe.
- You need 70B at q4 in VRAM: neither, alone. Both spill, and both land near 5 tok/s.
Side-by-side specifications and current listings are on the RTX 3090 vs RTX 4090 comparison page. For the tier below, where VRAM rather than speed is the deciding variable, see RTX 3060 12GB vs RTX 3090 and the RTX 3060 12GB local-LLM guide.
Citations and sources
- LocalScore — RTX 4090 accelerator page
- LocalScore — RTX 3090 accelerator page
- Hardware Corner — RTX 4090 LLM benchmarks
- Hardware Corner — RTX 3090 LLM benchmarks
- Kunal Ganglani — LLM benchmarks
- DatabaseMart — Ollama GPU benchmark, RTX 4090
- llama.cpp — performance discussions
- thc1006 — Qwen3.6 speculative decoding on an RTX 3090
- Spheron — RTX 5090 vs RTX 4090 inference benchmarks
- The Register — RTX 3090 AI benchmark
- GigaGPU — Llama 3 70B on an RTX 3090
- Mustafa.net — LLM tokens-per-second benchmarks
- PassMark — GeForce RTX 4090
- PassMark — GeForce RTX 3090
- TechPowerUp — GeForce RTX 4090 review
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
