Building a workstation around the RTX 5090 for local AI work comes down to four decisions: how much VRAM headroom you actually need, how you size CPU and system RAM around it, how you power a 575W card safely, and where you draw the line between "good enough" and over-building. This guide focuses on the parts-list and sizing math; for a deeper walkthrough of cooling and case selection specifically, see SpecPicks' companion piece, the RTX 5090 AI Build Guide: CPU, RAM, PSU & Cooling for Local Inference.
Quick answer
Per NVIDIA's official product specifications, the GeForce RTX 5090 ships with 32GB of GDDR7 memory on a 512-bit bus and a 575W total graphics power rating. For a local-AI-focused build, that translates to roughly:
- CPU: a modern 8-16 core desktop chip (data loading and preprocessing, not raw AI throughput, is where the CPU matters most)
- RAM: 64GB DDR5 as a comfortable baseline, 128GB if you're doing multi-modal or CPU-offloaded training
- PSU: 1000W-1200W, 80+ Gold or better, with enough 12V-2×6 or 8-pin headroom for the card's rated draw
- Case/motherboard: clearance for a triple-slot cooler and a PCIe 5.0 x16 slot
What components are essential for an RTX 5090 AI build?
| Component | What to target | Why it matters for AI workloads |
|---|---|---|
| GPU | RTX 5090, 32GB GDDR7 | VRAM capacity is the single biggest determinant of what model sizes you can run without offloading |
| CPU | 8+ cores, PCIe 5.0 support | Feeds the GPU with data during training/preprocessing; rarely the bottleneck for pure inference |
| RAM | 64GB-128GB DDR5 | Staging area for datasets, CPU-offloaded model layers, and multi-modal pipelines |
| Storage | NVMe SSD, 2TB+ | Model checkpoints and datasets are large; slow storage stalls data loading |
| PSU | 1000W-1200W, 80+ Gold | Card's rated 575W total graphics power needs headroom alongside CPU and drives |
| Case | Triple-slot clearance | The RTX 5090 Founders Edition and most partner cards are physically large |
This isn't a gaming-first parts list — a gaming-only RTX 5090 build can get away with less RAM and a smaller PSU margin. AI workloads, particularly anything that keeps a model resident in VRAM for hours at a time while also running a browser, IDE, and monitoring tools, benefit from the extra system RAM and power headroom.
How much VRAM do you actually need?
The RTX 5090's 32GB is the headline spec, and it's the number that determines what you can run without resorting to CPU offloading or multi-GPU setups. As a rough sizing framework (based on standard quantization math, not a specific benchmark run):
| Model size | 8-bit quantized (approx.) | 4-bit quantized (approx.) | Fits in 32GB? |
|---|---|---|---|
| 7B parameters | ~7GB | ~4GB | Comfortably, with room for a large context window |
| 13B-14B parameters | ~14GB | ~8GB | Comfortably |
| 30B-34B parameters | ~30GB | ~17-20GB | 4-bit yes; 8-bit is tight once you add context/KV cache overhead |
| 70B parameters | ~70GB | ~35-40GB | Generally no at 4-bit either — expect CPU offload or a multi-GPU setup |
These are back-of-envelope figures from standard quantization ratios, not a specific vendor benchmark, and actual VRAM usage varies with context length, KV cache size, and the inference engine's overhead — treat them as planning ranges, not guarantees. For a look at how a smaller local-LLM build handles this differently, see SpecPicks' comparison of an AMD Ryzen AI mini PC vs a DIY RTX 3060 build for local LLMs, which sits at the opposite end of the VRAM-budget spectrum from a 5090 build.
RTX 5090 vs MI300X: which is better for AI work?
This comparison shows up in search a lot, but it's mostly apples-to-oranges. Per AMD's official Instinct MI300X product page, the MI300X is a data-center accelerator designed for server and cloud deployment, not a desktop add-in card — it isn't something you install in a tower case next to a consumer motherboard. The RTX 5090, by contrast, is explicitly a consumer/prosumer GeForce card.
What actually differs for someone planning a desktop build:
- Software ecosystem: the overwhelming majority of open-source inference and training tooling (PyTorch, most quantization libraries, TensorRT-LLM) defaults to CUDA first, which favors NVIDIA hardware for day-one compatibility.
- Deployment model: MI300X is sold and deployed as server/cloud infrastructure; comparing it to a desktop GPU build is really a "local workstation vs. cloud/on-prem server" decision, not a same-category GPU shootout.
- Cost and availability: consumer RTX 5090 cards are available through normal retail channels; MI300X procurement runs through enterprise/OEM server channels.
If you're deciding between building a desktop AI rig and renting cloud accelerator time, that's a genuinely useful question — just don't expect a direct spec-for-spec comparison table between a GeForce card and a data-center accelerator to tell the whole story. For more on how AMD's consumer-facing AI hardware compares to a desktop GPU approach, see SpecPicks' AMD Ryzen AI Halo review roundup: AMD builds its own DGX Spark rival.
Optimizing the RTX 5090 for inference
Once the hardware is assembled, most of the remaining performance is a software problem:
- Use an optimized inference engine. NVIDIA's TensorRT-LLM (see NVIDIA's TensorRT developer page) is built specifically to optimize transformer inference on CUDA hardware, and is generally worth evaluating over a naive PyTorch inference loop for production-style local serving.
- Quantize deliberately. Dropping from 16-bit to 8-bit or 4-bit quantization is usually the single biggest lever for fitting larger models in 32GB, at some quality cost that varies by model and quantization method — test your specific use case rather than assuming a fixed quality/size tradeoff.
- Watch KV cache growth. Long-context sessions grow memory usage during generation, not just at load time; this is a common reason a model that "fit" at short context runs out of VRAM mid-conversation.
- Profile before you tune. Whether the bottleneck is GPU compute, VRAM bandwidth, or CPU-side data feeding varies by workload — profiling tools are the only reliable way to know which lever to pull.
If you're exploring what's runnable on more modest hardware before committing to a 5090-class build, SpecPicks' look at running an open-sourced Grok-class model locally on an RTX 3060 is a useful lower-budget reference point.
Rounding out the build: a budget monitor pairing
An AI workstation doesn't need a high-refresh gaming panel — most of the time you're watching logs, a terminal, or a monitoring dashboard rather than frame-pacing a game. A low-cost secondary display is a reasonable place to save budget versus splurging on a second premium monitor. Two budget 1080p options currently in SpecPicks' catalog:
| Product | Price | Notes |
|---|---|---|
| Sceptre 24-inch Prime Monitor | $102.97 | 1080p, 99% sRGB, dual HDMI + VGA — solid as a secondary/monitoring display |
| Sceptre Curved 24-inch Gaming Monitor | $82.97 | 1080p curved, 98% sRGB, dual HDMI + VGA — cheaper option if you want a curved secondary panel |
More from SpecPicks
If you're building out a broader hardware hobby beyond this AI rig, a few related builds worth a look:
- Build a period-correct Windows 98 gaming PC with CompactFlash storage
- CompactFlash-to-IDE boot drive: the silent Win98 build upgrade
- Ex-Sega engineer builds a "super-realistic" CRT emulator
- Build a Raspberry Pi 4 AirPlay receiver in 2026: full walkthrough
Citations and sources
- NVIDIA GeForce RTX 5090 official product page — VRAM capacity, memory bus, and total graphics power specifications
- NVIDIA TensorRT developer page — inference optimization tooling referenced in the optimization section
- AMD Instinct MI300X product page — data-center accelerator deployment model referenced in the comparison section
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
