Yes — and comfortably. A 4B-class coding model at Q4_K_M is roughly 2.5–3GB of weights, which leaves the MSI GeForce RTX 3060 Ventus 3X 12G with 9GB of headroom for a 32K–64K token context window plus KV cache. Paired with an eight-core AM4 chip like the Ryzen 7 5800X and a fast NVMe or SATA workspace drive like the Crucial BX500 1TB SSD, the whole edit-run-retry loop fits on one $700 rig.
Why 4B models changed the local coding-agent calculus
For two years the assumption was that "real" local coding needed at least a 32B model, ideally 70B, running on 24–48GB of VRAM. That put it out of reach for anyone without a used 3090 or a 4090. In late 2025 the picture shifted: purpose-trained 4B code models started hitting 85–90% pass rates on constrained coding benchmarks — SWE-bench-verified subsets, HumanEval, and MBPP variants. They are not frontier-level on open-ended architecture questions, but they are highly competent at focused edits, refactors, test generation, and single-file rewrites, which is what agent loops actually do most of the time.
That opens the door to running a coding agent locally on hardware you probably already own. A 4B model at Q4_K_M takes ~2.5GB VRAM; with KV cache for 32K context that swells to ~4–5GB. On a 12GB card that leaves 7GB free for tool-use scratch, embedding models running alongside, or a second small model doing planning while the coder does the edits. The economics also change: at zero per-token cost, you can afford the retry-heavy iterative style that agents use, without watching a metered bill tick up.
The catch is that not every 4B model is a good coding model. The strong ones are usually distilled from a larger teacher on a code-heavy dataset, then instruction-tuned on tool-use conversations. When someone claims "87% on our internal benchmark," take that at face value only until you run it on your own repo — code models tend to be brittle to codebase style and prompt shape.
Key takeaways
- A 4B coding model at Q4_K_M uses ~2.5GB VRAM; the RTX 3060 12GB has 9GB left for context and tools.
- Real-world single-user tok/s on a 3060 with a 4B model: 55–90 tok/s at short prompts, 30–50 tok/s at 8K–16K context.
- Storage matters more than most builders think — the agent constantly reads files and writes patches. A Crucial BX500 1TB is the cheap safe floor; NVMe is a real upgrade.
- 32GB DDR4-3600 is the practical minimum for running the agent plus a modern IDE, Docker, and your test suite.
- The local rig breaks even against a heavy cloud subscription in weeks, not months, at real agent volumes.
Why do 4B parameter models work for local coding agents?
Small does not mean weak when the model is trained specifically for the job. Modern 4B code models benefit from three trends: better data curation (deduped, license-checked, permissive code corpora), knowledge distillation from far larger teachers, and tool-use fine-tuning that teaches the model to call editors, test runners, and search directly. A well-trained 4B model can be surprisingly close to a general-purpose 30B model on constrained tasks like "rewrite this function to handle nulls" or "add a test for this branch."
Where the 4B model falls short is open-ended architecture: multi-file design decisions, cross-cutting refactors that require reasoning about how a change ripples through a codebase, and problems that require synthesis of many unstated constraints. Frontier models still win those. But the agent pattern — small, verifiable steps that a test or type checker can grade — plays to the 4B model's strengths. Every edit is a small verifiable delta; the model does not need to reason about the whole system, only the diff.
How much VRAM does a 4B agent leave free for context on a 12GB card?
Here is the memory math for a 4B model on the RTX 3060 12GB:
| Component | Q4_K_M | Q6_K | Q8_0 |
|---|---|---|---|
| Weights | 2.5 GB | 3.4 GB | 4.4 GB |
| KV cache @ 8K ctx (fp16) | 0.7 GB | 0.7 GB | 0.7 GB |
| KV cache @ 32K ctx (fp16) | 2.6 GB | 2.6 GB | 2.6 GB |
| KV cache @ 64K ctx (Q8 KV) | 2.6 GB | 2.6 GB | 2.6 GB |
| CUDA + runtime overhead | 0.6 GB | 0.6 GB | 0.6 GB |
| Total @ 32K ctx | 5.7 GB | 6.6 GB | 7.6 GB |
| Free for tools / second model @ 32K | 6.3 GB | 5.4 GB | 4.4 GB |
Even at Q8_0 (essentially lossless) plus a 32K context window, the 3060 12GB has 4–6GB free. That headroom lets you keep an embedding model resident for semantic code search, or run a small draft-verify pair (a 1B draft + the 4B verifier) for higher throughput.
What hardware runs the loop: GPU vs CPU responsibilities
The GPU runs the model. The CPU runs everything else — and there is a surprising amount of it. Every agent step involves file I/O, running a test suite, invoking a formatter, parsing a diff, updating a plan, and calling a linter. If any of those wait on a slow SSD or a starved CPU, the whole loop stalls.
| Component | Role | Why it matters |
|---|---|---|
| MSI RTX 3060 12GB | Model inference | 12GB is roomy for a 4B model; leaves headroom for a second small model. |
| AMD Ryzen 7 5800X | Agent runtime, tests, tool calls | Eight cores keep the test suite responsive while the model generates. |
| Crucial BX500 1TB | Repo, venv, model files | 500 MB/s sequential is plenty for the agent's file traffic; NVMe is a nice-to-have, not a requirement. |
| 32GB DDR4-3600 | OS + IDE + Docker + agent | Under 32GB you thrash swap the moment Docker + a test runner + the agent are all live. |
How fast is the edit-run-retry loop in practice?
The whole-loop wall-clock is the metric that matters. Below are typical numbers for a 4B model on the RTX 3060 12GB with a Ryzen 7 5800X and 32GB RAM, running a Node.js repo with a Vitest suite of ~200 tests:
| Step | Time |
|---|---|
| Read + tokenize file(s) into context (8K tokens) | 0.6 s |
| Model generates a patch (~400 tokens) | 4–7 s |
| Apply patch, run TypeScript check | 1.5 s |
| Run relevant Vitest tests | 3–8 s |
| Model reasons on failure and retries | 4–7 s |
| Total for one edit-run-verify loop | ~13–24 s |
The model's generation time is the biggest single chunk, but the test runner is a close second. Speeding up the loop means either shrinking the context, using a faster model (draft-verify pair), or improving the test selection so you run only the tests that touch changed files. Community benchmark posts on the llama.cpp issue tracker report similar wall-clock numbers for 4B code models on 12GB Ampere cards.
How does context length affect multi-file agent runs?
Agents that work across three or four files often want 16K–32K token contexts to keep the full picture in view. At 32K the 4B model's tok/s drops from ~85 (2K context) to ~35 (32K context). That is still workable for an agent loop, but it changes the calculus: at 32K each retry adds ~7–10 seconds of generation time. Aggressive context trimming — dropping older test failures, summarising unchanged files — pays back multiple seconds per iteration.
A pragmatic pattern: use 8K context for individual edits, and jump to 32K only when the plan step needs to see cross-file structure. Ollama and llama.cpp both let you switch model instances with different context settings, or you can run one hot with a large context and pay the tok/s cost consistently.
What SSD and RAM does the agent workspace need?
For most builders a 1TB Crucial BX500 SATA SSD is the right floor. It sustains 500 MB/s reads, which is far more than the agent will demand — the bottleneck is IOPS and latency on many small files, not raw throughput. If you have a repo full of thousands of small TS files, a real NVMe drive shaves seconds off git status, tsc, and test discovery.
RAM is where builders under-invest. 16GB looks fine on paper until you launch VS Code, a couple of Docker containers, a browser, and the agent. Then you swap and the whole loop stalls. 32GB DDR4-3600 in dual channel on an AM4 board is the smallest sane build for this workload. 64GB is nicer if you keep multiple long-running agents alongside your development environment.
Perf-per-dollar: local 4B agent vs cloud API cost over a month
Metered coding APIs in 2026 land around $3–$15 per million tokens depending on the tier. A heavy agent user issues millions of tokens a day — retry-heavy loops, big contexts, and tool-call transcripts add up. A modest 10M-tokens-per-day habit runs $900–$4,500 a month at the mid tier.
A fresh 3060 build — 3060 12GB, Ryzen 7 5800X, 32GB RAM, 1TB SSD, mid-tier B550, case, PSU — comes in under $900 in 2026 (used-parts market). Even accounting for electricity (~170W under load, ~$25/month at 12¢/kWh at 8 hours/day), the local rig pays for itself in under a month at heavy usage and continues to earn its keep with zero per-token cost. For light users the cloud is cheaper; for sustained agent loops the crossover is fast and clear.
Refer to the Ollama model library for available 4B code models; most publish per-model VRAM estimates that match the math above.
Bottom line: when a local 4B agent beats a cloud subscription
Go local if:
- You run agent loops sustainably (hours per day, not minutes), and metered token bills are a real line item.
- Privacy matters — your code should not leave your machine.
- You want deterministic latency, not "whatever the API queue is doing today."
- You already have most of the hardware and only need the GPU.
Stay on cloud if:
- Your usage is spiky and light.
- You need the strongest possible reasoning for architecture-level tasks.
- You value zero-maintenance uptime over cost.
Common pitfalls that stall the local agent loop
- Loading a general-purpose 4B model instead of a code-tuned one. A vanilla 4B chat model without code-heavy fine-tuning drops 15–25 points on the same tasks. The distinction matters more than the base weights.
- Running a slow tokenizer. Some Python tokenizer wrappers are 3–5× slower than a compiled Rust or C++ tokenizer. If your prefill wall-clock is dominated by tokenization, switch runtime.
- Serialising tool calls. Agents that run tests, formatter, and type check serially waste seconds per iteration. Parallelise the ones that do not depend on each other.
- Bloated context. Every retry that includes the last three failures grows the context; every context growth slows the next retry. Trim after every N failures.
- No context management. Long-running agents that never rewind end up in 32K prompts full of stale plan text. Summarise the plan every K steps.
- Under-provisioning RAM. 16GB looks fine at boot; then Docker + VS Code + the agent + a test runner + a browser hit swap simultaneously and the loop stalls. 32GB is the smallest sane build.
When NOT to go local for coding
The local coding agent story fits a specific profile: sustained daily use, code you cannot send to a vendor, and a repo shape the 4B model can actually help with (well-scoped edits, tests, refactors). It does not fit if your work is architecture-heavy, if you rarely code, or if your team already pays for a metered coding subscription that covers you comfortably. Do not build the rig because the maths looked good; build it because your workload actually maps onto it.
What to try first before buying hardware
If you already own a 12GB card, pull a 4B code model with Ollama today. Run it against your real repo for a week. Measure how often it gets edits right on the first try, how often it needs a retry, and how much your day slows down or speeds up. The best signal for whether local pays back is a week of your own numbers, not a blog post's benchmark.
Related guides
- Qwen 3.6 27B on a 12GB RTX 3060: Which Quant Actually Fits? — the 27B counterpoint to this 4B story.
- Ollama vs llama.cpp on a 12GB GPU — which runner best fits an agent loop.
- Best NVMe SSD for Local LLM Model Storage in 2026 — when the BX500 is not enough.
Citations and sources
- NVIDIA GeForce RTX 3060 spec sheet — TechPowerUp
- llama.cpp reference implementation and per-model quant tables — ggml-org/llama.cpp on GitHub
- Ollama model library, 4B code model line-up — Ollama library
