Budget Ollama Build Under $500: What Actually Runs in 2026

Step 0: which bottleneck are you buying against — VRAM, memory bandwidth, or idle power?

Every local-inference purchase is an answer to one of three questions, and buying against the wrong one is how people end up with hardware that benchmarks well and disappoints in use.

VRAM capacity is a gate. It is binary. A model either fits in the card's memory or it does not. If it does not, the runtime splits layers between GPU and system RAM, and the layers living in system RAM run at DDR4 bandwidth instead of GDDR6 bandwidth. There is no graceful degradation here — the moment you offload, per-token latency steps up sharply and stays there.

Memory bandwidth is a speed limit. Once the model fits, generation speed is governed almost entirely by how fast the card can stream weights out of memory. Each generated token requires reading essentially the whole active weight set. That gives a useful back-of-envelope ceiling: bandwidth ÷ model file size. With 360 GB/s of bandwidth (TechPowerUp) and a 5.03 GB q4_K_M 8B file, the theoretical ceiling is around 72 tokens per second. Real throughput lands meaningfully below the ceiling because of attention overhead, sampling, and framework costs, but the ratio between two configurations usually holds.

Idle power is a running cost. A machine that answers questions on demand spends most of its life idle. A 170 W board power figure is the ceiling under load, not the number you pay all day — but the platform underneath it draws power continuously, and that is the figure that compounds over a year of uptime.

For a sub-$500 build, the honest ranking is: capacity first, bandwidth second, idle power third. Capacity decides what you can run at all. Bandwidth decides whether it is pleasant. Idle power decides whether you leave it on.

What does 12GB of VRAM actually hold in 2026?

The table below uses published file sizes from Qwen's official GGUF repositories, which is the cleanest public source for "how big is this actually" because the quantizations are released by the model authors rather than reconstructed by a third party. "Fits in 12 GB" means weights resident with room left for a working context window.

File sizes: Qwen3-8B-GGUF, Qwen3-14B-GGUF, Qwen3-30B-A3B-GGUF. Bandwidth ceiling is arithmetic on the 360 GB/s figure, not a measured result — treat it as an upper bound that no real runtime reaches.

Three things fall out of that table. First, the 8B tier is entirely comfortable at any quantization level including Q8_0, so quality loss from quantization is a choice rather than a constraint. Second, the 14B tier is where 12 GB starts to bind: Q4_K_M fits with context, Q5_K_M is marginal, and anything above that is out. Third, the 30B-class MoE models do not fit at any published quantization, which is the honest answer to the most common question asked about this card.

Note also what is missing from Qwen's official repositories: there are no Q2_K or Q3_K_M builds published there. Sub-4-bit quantizations exist in the wider community, but they are third-party reconstructions, and the quality trade at that level is workload-specific enough that no table can tell you whether it is acceptable for your prompts.

Which GPU goes in the $500 build?

Two cards dominate the used listings for this tier, and both are the same silicon with different coolers.

The ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB is the shorter card, which matters if you are building into a small case or a repurposed office desktop. The MSI GeForce RTX 3060 Ventus 2X 12G is longer with more fin area, which generally means lower fan speed at the same heat load — relevant for a machine that sits in the room you work in. For inference, the performance difference between them is noise: identical memory subsystem, identical shader count, and factory clock differences that affect compute the workload is not bound by.

Buy on cooler and physical fit, not on the OC badge. And read the memory figure on the listing carefully — the RTX 3060 was also sold in an 8 GB variant on a 128-bit bus, and that card is a different product for this purpose. It cannot hold the 14B tier that justifies buying a 3060 in the first place. Our RTX 3060 12GB vs 8GB comparison covers why the two get confused so often.

Can you skip the GPU entirely?

The AMD Ryzen 5 5600G is a six-core, twelve-thread part with integrated Radeon graphics (AMD), and it is the reason a GPU-free build is even worth discussing. It boots without a discrete card, which means the same chip can be the host in a GPU build or the whole machine in a CPU-only one.

The physics are unforgiving, though. CPU-side inference streams weights from system RAM, and a dual-channel DDR4-3200 configuration provides on the order of 51 GB/s of theoretical bandwidth — roughly one-seventh of the 3060's 360 GB/s (TechPowerUp). Applying the same bandwidth ÷ file-size ceiling to a 5.03 GB q4_K_M 8B model gives a theoretical upper bound around 10 tokens per second, and real throughput sits below that.

That is not useless. For a background summarizer, a classification job, or an overnight batch task, ten tokens per second is fine. For interactive chat it is uncomfortable, and for a coding assistant that has to re-read a file before it answers, it is worse than uncomfortable — because prompt processing on the CPU is slower still.

The rule: CPU-only is a legitimate starting point if you already own the chip. It is a poor place to spend new money if inference is the goal. Run one matched pair of DIMMs rather than a single stick either way — a single-channel configuration roughly halves effective bandwidth, and bandwidth is the entire game here.

Prefill vs generation: why the 3060 feels fast on chat and slow on long documents

Local models do two different jobs with the same weights, and they scale differently.

Prefill — also called prompt processing — reads your entire input and builds the key/value cache. It is compute-bound and parallel: the model can process many tokens at once, so throughput is governed by the card's arithmetic rate, not its memory bandwidth. With 3,584 CUDA cores (NVIDIA), the 3060 has enough compute to make short prompts feel instantaneous.

Generation — producing the answer — is sequential and memory-bound. Every single token requires streaming the weight set again. This is where the 360 GB/s figure sets the pace.

The practical consequence: a chat turn with a 200-token prompt spends almost no time in prefill and all of its time in generation, so the card feels responsive. Paste in a 20,000-token document and prefill becomes the dominant cost — you sit and wait before the first token appears, then generation proceeds at its usual rate. Users read this as "the model got slow," but nothing changed about generation; a different phase started dominating.

If your workload is document-heavy, prefill throughput matters more to your experience than tok/s does, and that is a compute question rather than a VRAM one.

How much context can you afford?

The KV cache grows linearly with context length and it competes with the weights for the same 12 GB. Its size is computable from the model's architecture: 2 (keys and values) × layers × KV heads × head dimension × context length × 2 bytes at fp16.

For Qwen3-8B — 36 layers, 8 KV heads, head dimension 128 — that is 144 KiB per token. For Qwen3-14B — 40 layers, 8 KV heads, head dimension 128 — it is 160 KiB per token.

Architecture figures from the published model configs; cache sizes are arithmetic on them.

Read the last column carefully. A 14B model at Q4_K_M fits on a 12 GB card at 4K and 8K context. At 16K it is uncomfortably tight once you account for the runtime's own allocations. At 32K it does not fit at all, and the runtime will start offloading layers — which is the cliff described in Step 0.

Two mitigations are worth knowing. Quantizing the KV cache to 8-bit roughly halves those figures, at some cost to long-context recall. And dropping to the 8B tier buys both a smaller weight footprint and a smaller per-token cache, which is why 8B models feel so much more comfortable on this card at long context than the parameter difference alone suggests.

Where do the models live?

Model weights sit on disk and get read into memory once per load. That makes storage a capacity question with a one-time latency cost attached, not a throughput bottleneck.

The Crucial BX500 1TB reads at up to 540 MB/s (Crucial). At that rate, a 5.03 GB 8B model loads in roughly 9 seconds and a 9 GB 14B model in roughly 17 seconds, using the published file sizes from Qwen's GGUF repositories. That cost is paid once when you switch models, then never again while the model stays resident.

The Kingston 960GB A400 is the smaller-library alternative — same SATA interface, same "fast enough for weights" characteristic, less room for a collection. A terabyte holds a lot of GGUF files; 960 GB holds slightly fewer. Neither is the constraint on this build.

Where SATA genuinely hurts is swap. If you run out of system RAM and the OS starts paging, a SATA drive turns a slow situation into an unusable one. That is a configuration problem — fix it with RAM, not with a faster drive. Our NVMe vs SATA comparison for model libraries works through the case where NVMe does earn its price premium.

What about a Raspberry Pi 4 Model B instead?

The 8 GB Raspberry Pi 4 Model B is a quad-core Cortex-A72 at 1.5 GHz with LPDDR4-3200 SDRAM (Raspberry Pi datasheet). It is a genuinely capable little computer and a genuinely poor inference host, for the same reason the 5600G is limited but more so: memory bandwidth.

The honest ceiling is the 1-4B class at aggressive quantization, used for narrow tasks — a classifier, a small summarizer, a voice-assistant intent parser. A 7-8B model at q4_K_M is 5.03 GB (Qwen3-8B-GGUF), which technically fits in 8 GB of unified system memory but leaves the operating system almost nothing and runs at a rate measured in seconds per token rather than tokens per second.

Buy a Pi for the thing a Pi is good at: always-on, silent, single-digit-watt orchestration. Point it at a model running somewhere else. It makes an excellent front end for an Ollama endpoint hosted on the 3060 box.

Perf-per-dollar and perf-per-watt

Three configurations, compared on the axes that matter for an always-on box. Bandwidth figures are the published specs; the throughput column is the bandwidth ÷ file-size ceiling applied to a 5.03 GB q4_K_M 8B model, which is an upper bound rather than a measurement.

Bandwidth sources: TechPowerUp for the GPU, AMD for the APU platform, Raspberry Pi for the SBC.

The electricity arithmetic is simpler than it looks. The 170 W figure is a ceiling reached only while generating; a box that answers a few dozen queries a day spends the overwhelming majority of its hours idle, and idle draw is a platform property rather than a GPU one. Two levers actually move the annual number: a power limit on the card, which trades a small amount of throughput for a large reduction in peak draw, and suspending the machine between sessions, which is usually the bigger saving of the two.

Common pitfalls

• Buying the 8 GB RTX 3060. Same name, 128-bit bus, and no room for the 14B tier. Read the listing's memory figure, not the model number.
• Running a single RAM stick. Halves effective bandwidth for anything offloaded to the CPU and costs nothing to fix at build time.
• Under-sizing the PSU. NVIDIA's recommendation is a 550 W system supply for this card (NVIDIA). A tired unit salvaged from an old build is the most common cause of unexplained crashes under sustained load.
• Setting context to the model maximum by default. A 32K window on a 14B model costs 5 GiB of KV cache and will push you off the card. Set the context you actually use.
• Assuming a used mining card is unsafe. The parts that wear are fans and thermal pads, not the die. Check loaded VRAM temperatures in the first week and budget for a fan replacement rather than avoiding the category.

When NOT to build this

Skip the 12 GB tier and save if any of these describe you. Your workload is 27B-and-up models — the 30B-A3B Q4_K_M file alone is 18.6 GB (Qwen3-30B-A3B-GGUF), which no amount of tuning fits in 12 GB. You run a coding agent that holds a large repository in context, where the KV-cache table above turns into a hard wall at 16K. Or you intend to fine-tune rather than infer, which needs optimizer state alongside weights and blows past 12 GB immediately.

In all three cases, the cheap entry becomes an expensive detour. Buying twice costs more than waiting once.

Complete the build

Street prices on used GPUs and DDR4 move constantly, so the running total is a moving target — check current listings before committing. If you land over budget, cut in this order: drop from 32 GB to 16 GB of RAM (recoverable later), take the 960 GB drive instead of the 1 TB, and accept a plainer case. Do not cut the GPU to an 8 GB card to make the number work. That is the one component where the compromise changes what the machine can do rather than how pleasant it is to use.

Bottom line

Buy the used RTX 3060 12GB, put it on a 5600G with two sticks of DDR4-3200, and give it a 1 TB SATA drive for the model library. That configuration runs the 8B tier at any quantization with room to spare and the 14B tier at Q4_K_M with 8K of context, which covers chat, retrieval over a modest document set, and 7-14B coding assistance.

Save for a 16 GB or 24 GB card instead if your real workload is 27B-class models, long-context agents, or fine-tuning. The 12 GB ceiling is generous for what it is and completely rigid about what it is not.

Related guides

• RTX 3060 12GB Local LLM Guide: Which Models Actually Fit
• Best GPU for Ollama Under $300: Why 12GB Beats a Faster 8GB Card
• Ollama vs vLLM vs llama.cpp on a 12GB GPU
• RTX 3060 12GB vs Ryzen 5 5600G iGPU
• RTX 3060 benchmark data

Citations and sources

• NVIDIA — GeForce RTX 3060 family specifications (accessed 2026-09-09)
• TechPowerUp — GeForce RTX 3060 12 GB database entry (accessed 2026-09-09)
• Ollama — project repository (accessed 2026-09-09)
• Hugging Face — Qwen3-8B-GGUF file sizes (accessed 2026-09-09)
• Hugging Face — Qwen3-14B-GGUF file sizes (accessed 2026-09-09)
• Hugging Face — Qwen3-30B-A3B-GGUF file sizes (accessed 2026-09-09)
• Hugging Face — Qwen3-8B model card and config (accessed 2026-09-09)
• Hugging Face — Qwen3-14B model card and config (accessed 2026-09-09)
• AMD — Ryzen 5 5600G product page (accessed 2026-09-09)
• Crucial — BX500 SATA SSD product page (accessed 2026-09-09)
• Raspberry Pi — Raspberry Pi 4 Model B datasheet (accessed 2026-09-09)

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.

Prices move constantly on used hardware — check the current listing before buying. As an Amazon Associate, SpecPicks earns from qualifying purchases.