gpt-oss 120B at Home: RTX 3060 12GB Offload vs Ryzen 7 5800X CPU-Only

Who is running a 120B model at home, and why it is possible at all

Two years ago "120B at home" meant a rack of used datacenter cards. In 2026 it means a mid-range gaming PC with a lot of RAM, and the reason is architectural. gpt-oss-120b is a mixture-of-experts model: 116.8B total parameters, but only 5.1B are active for any given token, because a router picks the top 4 of 128 experts in each of its 36 layers (OpenAI — gpt-oss model card). OpenAI also shipped the expert weights — more than 90% of the parameter count — already quantized to MXFP4 at 4.25 bits per parameter, which is how the whole model fits on a single 80 GB GPU.

That changes the hardware math versus a dense 70B. A dense model reads every weight for every token, so it needs every weight in fast memory. A MoE model only reads the chosen experts, so the bulk of the weights can sit in slower, cheaper system RAM while the small, always-used parts — attention, the router, the embeddings — live on the GPU. llama.cpp exposes this directly with --n-cpu-moe, which keeps a chosen number of layers' expert weights on the CPU (llama.cpp — running gpt-oss guide).

The people doing this are running private document synthesis, offline evaluation against hosted models, or simply want the strongest open-weight reasoning model that fits a desktop. For them there are two realistic paths on an existing AM4 platform: add a 12 GB ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB and split the model, or skip the GPU and run everything on an AMD Ryzen 7 5800X with a large RAM kit. This synthesis compares the two.

Key takeaways

• Quantization will not save you. Every GGUF of this model lands between 62.6 GB (Q2_K) and 65.4 GB (F16) (Unsloth — gpt-oss-120b-GGUF), because the experts are natively MXFP4.
• 64 GB of system RAM is the floor for the GPU build. With all experts on the CPU, a 12 GB card holds about 6 GB of weights and system RAM holds about 60 GB, per Crook's build.
• RAM bandwidth sets generation speed. One RTX 4070 12 GB build went from 9.72 to 28 tok/s generation when its DDR5 was fixed from 2,000 MT/s to rated speed (carteakey — optimizing gpt-oss-120b).
• The GPU wins prompt processing. GPU-assisted builds with RAM at rated speed publish 229–428 tok/s of prompt processing; a 24-core EPYC server running CPU-only measured 108.56 tok/s (llama.cpp issue #17936).
• The KV cache is small. Arithmetic from the model's config puts it near 36 KB per token, about 1.2 GB at 32K context.

Step 0: which bottleneck are you actually buying around?

Before spending anything, work out which of three limits you hit first. Each one has a different fix, and buying the wrong part is how people end up with a 24 GB card that is no faster than a 12 GB one.

If you have neither 64 GB of RAM nor a way to get it, stop here and read the verdict matrix's third option.

VRAM and quantization requirements

The table below is the counterintuitive part. File sizes are from the Unsloth and ggml-org GGUF repositories; the fit judgments assume a 12 GB GPU and leave room for the operating system.

Unsloth's own guidance matches: at least 66 GB of combined VRAM and RAM, or 66 GB of system RAM, for 6+ tokens per second (Unsloth — how to run gpt-oss). Pick MXFP4 — the checkpoint is listed at 60.8 GiB in the model card, the format the model was trained in — and spend your effort on memory instead.

Spec delta: the two builds side by side

Both builds share the same AM4 host so the comparison isolates the GPU. AMD lists the Ryzen 7 5800X at 8 cores and 16 threads, 32 MB of L3, 105 W TDP, PCIe 4.0 and DDR4 up to 3,200 MT/s — and notes that a discrete graphics card is required (AMD — Ryzen 7 5800X).

GPU figures are from NVIDIA's specification (12 GB GDDR6, 192-bit, 170 W — NVIDIA — RTX 3060 family) and the RTX 30-series reference table (360 GB/s, $329 launch — Wikipedia — GeForce RTX 30 series); TechPowerUp's database carries the same GA106 configuration (TechPowerUp — RTX 3060 12 GB). Dual-channel DDR4-3200 is 51.2 GB/s theoretical (Tom's Hardware forum — DDR4-3200 bandwidth), or 25.6 GB/s per channel (Wikipedia — DDR4 SDRAM).

Benchmark table: tok/s across configurations

No public source measures gpt-oss-120b tok/s on an RTX 3060 specifically, or CPU-only on a dual-channel DDR4 desktop. The table collects the closest published measurements, labelled by exactly what was measured, so you can place the two builds between them.

How to read it for the two builds here:

• RTX 3060 12 GB + 64 GB DDR4. The feasibility is confirmed — a 3060 runs the Q4_K_M file with every expert on the CPU at a 131K context in the llama.cpp issue above. The closest speed analog is Crook's 12 GB Ampere card on DDR4, and because generation there is bound by system RAM rather than the GPU, treat 18–22 tok/s as the upper bound for a 3060 on slower DDR4-3200.
• Ryzen 7 5800X CPU-only. No published number. The arithmetic ceiling is in the next section; the practical expectation is single-digit tok/s.
• The DDR4 EPYC outlier. The Hardware Corner run at 1.61 tok/s shows what a poorly matched offload split costs — the same article's plain layer split was slower still. Configuration matters as much as parts.

Prefill vs generation: why the GPU wins one phase and not the other

Every request runs in two phases. Prefill processes the whole prompt at once; it is a big matrix-multiply job, so it is compute-bound and loves a GPU. Generation produces one token at a time and has to read the active weights from memory for each token; it is bandwidth-bound.

In the GPU build, the attention layers and the KV cache sit on the RTX 3060, so prefill runs at GPU speed — the GPU-assisted rows above with RAM at rated speed publish 229–428 tok/s. In the CPU-only build, prefill runs on eight cores. The only CPU-only measurement available, a 24-core server, managed 108.56 tok/s; an eight-core desktop has a third of those cores. Feed a 4,000-token document into the CPU-only build and the wait for the first word becomes the dominant cost.

Generation is where the GPU helps less than people expect. The experts live in system RAM in both builds, so both are gated by the same 51.2 GB/s of dual-channel DDR4. The carteakey build proves the point from the other side: tripling memory bandwidth tripled token generation, with the GPU unchanged.

Here is the ceiling arithmetic, derived from cited inputs rather than measured. The model card lists 5.13B active parameters with MXFP4 experts at 4.25 bits, so each token reads on the order of 2.7 GB of weights. Divide 51.2 GB/s by 2.7 GB and the theoretical ceiling for a dual-channel DDR4-3200 desktop is about 19 tok/s. Real systems sustain around 84% of theoretical memory bandwidth in Microway's comparison (Microway — memory bus characteristics), and llama.cpp's CPU path lands well below even that: the 12-channel EPYC above has about 460 GB/s theoretical and produced 35.75 tok/s. Scale that efficiency down to two channels and a CPU-only 5800X lands in the single digits.

How does context length change the answer?

gpt-oss-120b's KV cache is unusually small for its size. The config lists 36 layers, of which 18 use a 128-token sliding window and 18 use full attention, with 8 KV heads of dimension 64 (Hugging Face — gpt-oss-120b config.json). Only the full-attention layers grow with context, which works out to about 36 KB per token at 16-bit precision — the sliding layers add a fixed few megabytes.

So on this model, long context rarely forces you back to system RAM. The practical limit is compute buffers: the RTX 3060 in the llama.cpp issue crashed at a micro-batch of 4,096 until a fix landed, while -ub 1024 -b 1024 ran normally. Keep the micro-batch modest on a 12 GB card, and spend leftover VRAM by lowering --n-cpu-moe so a few layers' experts move onto the GPU — the carteakey build packed 9.7 GB of weights onto its 12 GB card that way.

Does more system RAM beat more VRAM here?

Up to 64 GB, capacity is everything: below it the model does not load with a 12 GB card, and Crook's build notes flatly that 64 GB of system RAM is the minimum. Past that point, bandwidth matters more than size. AM4 is a two-channel platform — AMD's spec for the AMD Ryzen 9 3900X lists 2 memory channels at up to 3,200 MT/s (AMD — Ryzen 9 3900X specifications) — so no AM4 chip escapes the 51.2 GB/s ceiling at rated speed. That is why the 12-core 3900X should not generate meaningfully faster than the 8-core 5800X on this model: both drink from the same two channels.

Memory speed is the lever you do control. Tom's Hardware's review recommends DDR4-3200 as the base spec for the 5800X (Tom's Hardware — Ryzen 7 5800X review), and Phoronix's Zen 2 memory-scaling work notes AMD's own DDR4-3733 "sweet spot" recommendation (Phoronix — Ryzen 9 3900X memory scaling). Confirm in the BIOS that the kit is actually running at its rated speed — the carteakey build lost two-thirds of its generation speed to a kit stuck at 2,000 MT/s.

Choose 128 GB over 64 GB when you want the CPU-only fallback, want to keep other services running, or plan to keep more than one large model resident. Choose faster 64 GB over slower 128 GB if the GPU build is your only mode.

Storage matters because the checkpoint is not small. The Crucial BX500 1TB is rated at 540 MB/s read with 360 TB of endurance (Crucial — BX500 product flyer); at that rate the 63.4 GB file loads in about two minutes, and 1 TB holds it alongside several smaller models.

What about the CPU cooler and 24/7 thermals?

CPU-only inference pins all eight cores for as long as tokens are flowing, and the 5800X ships without a cooler. Tom's Hardware's review recommends a 280 mm AIO or an equivalent air cooler for it. The Noctua NH-U12S is the quiet-build answer rather than the maximum-headroom one: Noctua's infosheet lists a 158 mm tall single tower with a 120 mm NF-F12 fan, 22.4 dB(A) maximum noise, AM4 support and a 6-year warranty (Noctua — NH-U12S infosheet).

The trade-off favors quiet here. Token generation is memory-bound, so a few hundred megahertz of sustained boost clock lost to a smaller cooler costs very little in tok/s — while a box that runs overnight next to where people sleep benefits from a 22 dB(A) fan ceiling. In the GPU build the CPU works less hard per token, which makes the single tower an even easier call.

Performance per dollar and per watt

Start with what the GPU buys. The RTX 3060 launched at $329 and adds 170 W of board power. It buys prompt processing in the hundreds of tok/s instead of CPU-bound prefill, and it lets 64 GB of RAM do the job that CPU-only needs 128 GB for — so part of its cost comes back out of the RAM budget. Check the current street prices on the product pages linked throughout; they move weekly.

On running cost, US residential electricity averaged 18.34 ¢/kWh in June 2026 (EIA — Electric Power Monthly, Table 5.6.A). Worst case, with the GPU at its full 170 W for eight hours a day, that is about 496 kWh a year, or roughly $91. Under MoE offload the GPU rarely runs at full power, so real figures are lower — measure at the wall. Idle draw varies too much by motherboard and PSU to quote a single number; an always-on build should be checked with a plug-in power meter before you commit to 24/7.

The per-token view favors the GPU build on long prompts and roughly ties on short chat turns, because short turns are generation-dominated and both builds share the same DDR4 ceiling.

Common pitfalls

• Quoting the 20B number. A widely repeated "60 tok/s on an RTX 3060 with a Ryzen 7 5700X and 32 GB of DDR4" figure is a gpt-oss-20b result (llama.cpp discussion comment); the same poster says they would need 64 GB for the 120B.
• Expecting a smaller quant to fit 32 GB. Q2_K is still 62.6 GB.
• RAM below rated speed. XMP/EXPO off, or an unstable kit that has fallen back, silently cuts generation speed in proportion to bandwidth.
• Oversized micro-batches on 12 GB. Start at -ub 1024 and raise it only once the model loads and runs.
• No display output on a CPU-only 5800X. It has no integrated graphics, so plan for a spare display card or a headless install.

Verdict matrix

Get the RTX 3060 12GB offload build if… you already have or can buy 64 GB of DDR4, your prompts are long (documents, code, retrieved context), and you want the first token in seconds rather than minutes. This is the configuration published builds have proven.

Get the Ryzen 7 5800X CPU-only build if… you already own 128 GB of DDR4 and no suitable GPU, your jobs run as overnight batches, and prompt latency does not matter. Expect single-digit tok/s generation.

Skip gpt-oss 120B entirely and run gpt-oss-20b or a 14B dense model if… you want interactive chat or coding assistance, or you have only 32 GB of RAM. A model that fits entirely in 12 GB of VRAM will feel dramatically faster on the same card.

Bottom line

Buy the ZOTAC RTX 3060 Twin Edge OC 12GB, pair it with 64 GB of DDR4-3200 running at rated speed on a Ryzen 7 5800X, and run the MXFP4 GGUF in llama.cpp with --n-cpu-moe set just high enough that the model loads — then lower it step by step until VRAM is full. That build puts prefill on the GPU, keeps generation as fast as dual-channel DDR4 allows, and leaves an upgrade path to 128 GB if you later want the CPU-only fallback or longer sessions.

Live price comparison

• Ryzen 7 5800X vs Ryzen 9 3900X — live prices and specs
• ZOTAC RTX 3060 Twin Edge OC 12GB — current price
• RTX 3060 benchmark data · Ryzen 7 5800X benchmark data

Prices change frequently — the price shown on the retailer page at the time of your visit is authoritative. As an Amazon Associate, SpecPicks earns from qualifying purchases.

Related guides

• Best GPU for local LLMs in 2026
• RTX 3060 12 GB for local LLMs
• Best hardware for running MoE LLMs locally
• CPU offload for local LLMs: Ryzen 7 5800X with 12 GB of VRAM

Citations and sources

• OpenAI — gpt-oss model card (PDF) — 116.8B total / 5.1B active parameters, 36 layers, 128 experts with top-4 routing, MXFP4 at 4.25 bits, 60.8 GiB checkpoint (accessed 2026-09-10)
• Hugging Face — gpt-oss-120b config.json — layer types, KV heads, head dimension, sliding window (accessed 2026-09-10)
• ggml-org — gpt-oss-120b-GGUF — MXFP4 GGUF at 63.4 GB (accessed 2026-09-10)
• Unsloth — gpt-oss-120b-GGUF — Q2_K through F16 file sizes (accessed 2026-09-10)
• Unsloth — how to run gpt-oss — 66 GB memory guidance (accessed 2026-09-10)
• llama.cpp — running gpt-oss guide — --n-cpu-moe usage (accessed 2026-09-10)
• llama.cpp discussion — RTX 3090 + DDR5-5200 llama-bench — 26.90 tok/s generation, 364.80 tok/s pp2048 (accessed 2026-09-10)
• llama.cpp discussion — RTX 5070 + 128 GB DDR5 — ≈15 tok/s at 32K context (accessed 2026-09-10)
• llama.cpp discussion — gpt-oss-20b on Ryzen 7 5700X — the 60 tok/s figure is a 20b result (accessed 2026-09-10)
• llama.cpp issue #17936 — EPYC 9454P CPU-only, 108.56 / 35.75 tok/s (accessed 2026-09-10)
• llama.cpp issue #17931 — gpt-oss-120b on an RTX 3060 with experts on CPU at 131K context (accessed 2026-09-10)
• David Crook — gpt-oss-120b on a 12 GB card — 18–22 tok/s, ~6 GB VRAM / ~60 GB RAM, 64 GB minimum (accessed 2026-09-10)
• carteakey — optimizing gpt-oss-120b local inference — 428 / 28 tok/s tuned; 82.11 / 9.72 tok/s at 2,000 MT/s (accessed 2026-09-10)
• Hardware Corner — offloading gpt-oss MoE layers — 229.07 / 1.61 tok/s on DDR4-3200 (accessed 2026-09-10)
• NVIDIA — RTX 3060 family specifications — 12 GB GDDR6, 192-bit, 170 W (accessed 2026-09-10)
• Wikipedia — GeForce RTX 30 series — 360 GB/s, $329 launch (accessed 2026-09-10)
• TechPowerUp — GeForce RTX 3060 12 GB — GA106 configuration (accessed 2026-09-10)
• AMD — Ryzen 7 5800X — 8C/16T, 105 W, DDR4-3200, discrete GPU required (accessed 2026-09-10)
• AMD — Ryzen 9 3900X specifications — 12C/24T, 2 memory channels, DDR4-3200 (accessed 2026-09-10)
• Tom's Hardware — Ryzen 7 5800X review — DDR4-3200 base spec, 280 mm AIO-class cooling recommendation (accessed 2026-09-10)
• Phoronix — Ryzen 9 3900X memory scaling — AMD's DDR4-3733 recommendation (accessed 2026-09-10)
• Tom's Hardware forum — DDR4-3200 bandwidth — 51.2 GB/s dual-channel (accessed 2026-09-10)
• Wikipedia — DDR4 SDRAM — 25.6 GB/s per DDR4-3200 channel (accessed 2026-09-10)
• Microway — memory bus characteristics — typical vs theoretical bandwidth (accessed 2026-09-10)
• Noctua — NH-U12S infosheet — 158 mm, NF-F12, 22.4 dB(A) (accessed 2026-09-10)
• Crucial — BX500 product flyer — 540 MB/s read, 360 TB endurance (accessed 2026-09-10)
• EIA — Electric Power Monthly, Table 5.6.A — 18.34 ¢/kWh residential, June 2026 (accessed 2026-09-10)

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.