Set Up a Local LLM Coding Assistant in VS Code on an RTX 3060 12GB (2026)

To run a local LLM coding assistant in VS Code on an RTX 3060 12GB, install Ollama, pull a coder model that fits in 12 GB of VRAM (Qwen2.5-Coder 7B at q4 or DeepSeek-Coder-V2 Lite at q4 are the sweet spots as of 2026), then install the Continue.dev extension in VS Code and point its models config at http://localhost:11434. Enable a smaller model for autocomplete and the larger one for chat. That is the whole recipe — the rest of this guide is model sizing, wiring, and the perf tradeoffs.

Why this matters: a self-hosted Copilot alternative in 2026

The pitch for a local coding assistant in 2026 is not that it beats GPT-4-class cloud models on hard reasoning — it does not. The pitch is that it beats no assistant at zero marginal cost, keeps your code out of a vendor training corpus, and works on a flight. GitHub Copilot Business runs $19/user/month and Copilot Enterprise $39, and both send your prompts and surrounding files to a shared cloud service. For a hobbyist, a consultant with NDA'd client code, or a small team paying for five seats, the annualized cost of a used 12 GB Ampere card is recouped in months.

The RTX 3060 12GB is the odd hero of this story. It shipped in 2021 as a mid-range gaming card, but the 12 GB VRAM buffer — larger than the RTX 3070, 3070 Ti, 4060, 4060 Ti 8 GB, and 4070 8 GB variants — makes it the cheapest current-supply NVIDIA card that can hold a modern 7B-parameter coder model quantized to 4-bit plus a KV cache large enough to be useful. Per the TechPowerUp GPU database, the card provides 360 GB/s of memory bandwidth and 12,741 GFLOPs of FP32 compute across 3,584 CUDA cores on the GA106 die — modest by 2026 standards, but memory-bandwidth-bound inference workloads care more about the 12 GB buffer than the raw FLOPS.

Community reports from r/LocalLLaMA throughout 2025 and 2026 consistently place the RTX 3060 12GB as the "starter" recommendation for local code models, and the model landscape has finally caught up: Qwen2.5-Coder 7B, DeepSeek-Coder-V2 Lite 16B MoE, and StarCoder2 7B are all small enough to fit and strong enough to matter. That combination — cheap card, mature model tier, mature tooling — is what changed between 2024's "cute demo" and 2026's "actually part of my workflow".

Key takeaways

• The RTX 3060 12GB is the price-per-VRAM sweet spot for local coder LLMs; the 12 GB buffer holds a 7B model at q4 with headroom for a long context KV cache.
• Ollama plus the Continue.dev VS Code extension is the shortest path from zero to a working self-hosted Copilot alternative — one binary, one extension, one config file.
• Qwen2.5-Coder 7B (q4 or q5) is the default recommendation for chat and edit; a smaller 1.5B-3B model handles autocomplete latency better.
• Expect roughly 40-60 tokens/second on chat generations and sub-second first-token latency for autocomplete when everything is sized correctly.
• Local models do not replace GPT-4-class cloud models for large-refactor or novel-algorithm work; treat local as autocomplete and boilerplate, cloud as senior-engineer escalation.

What you'll need

A working setup on this budget is four pieces of hardware and one VS Code extension. The reference build below is deliberately conservative — none of these parts are cutting-edge, and all four can be sourced used or refurbished without hurting the experience.

• GPU: MSI GeForce RTX 3060 Ventus 3X 12G — the triple-fan Ventus 3X variant is the quiet default. The 12 GB buffer is the entire reason this card exists in a 2026 local-LLM guide; do not substitute an 8 GB card unless you enjoy VRAM-thrash and 5 tok/s.
• CPU: AMD Ryzen 7 5700X — 8 cores, 65 W, drop-in on any AM4 board. The CPU barely matters for GPU-resident inference, but you want enough cores to run VS Code, a browser, and a language server while Ollama holds the model in VRAM.
• Fast SSD for model storage: Samsung 970 EVO Plus 250GB NVMe — model files are 4-10 GB apiece and load into VRAM on first use. Per Tom's Hardware's 970 EVO Plus review the drive sustains ~3,300 MB/s reads, which means a 7B q4 model loads into VRAM in roughly one to two seconds after the first cold start. That matters when you're iterating on which model to keep resident.
• Bulk SATA SSD for the OS and projects: Crucial BX500 1TB — the NVMe slot is precious; put Windows or Linux and your workspace on the cheap SATA drive and reserve the NVMe for models. A BX500 is fine for source code and Docker layers.
• VS Code extension: Continue.dev is the default recommendation in 2026 — it has native Ollama support, an autocomplete pipeline separate from chat, and is actively maintained. Cline (formerly Claude Dev) is the alternative if you want an agentic workflow, but it is heavier and expects a more capable model.

Beyond the parts list, you want at least 32 GB of system RAM (16 GB works but is tight once Docker and Chrome are in the mix), an 80 Plus Gold 650 W PSU, and a case with real airflow — an RTX 3060 pulls 170 W under sustained inference and a Ryzen 7 5700X pulls 65 W, so you're dumping ~250 W of heat during a long chat session.

Which coder models fit in 12 GB?

The 12 GB VRAM budget on the RTX 3060 has to hold three things simultaneously: the model weights, the KV cache (memory of the current conversation), and a small amount of framework overhead. As a rule of thumb, plan for roughly 1.5-2 GB of KV cache at 8k context and leave ~1 GB of headroom. That gives you a ~9-10 GB weight budget.

The table below summarizes the popular 2026 coder models at typical quantization levels. VRAM figures are approximate steady-state usage at 8k context, drawn from the model cards and community reports on r/LocalLLaMA; tok/s ranges are the community-reported band for a stock RTX 3060 12GB running Ollama.

Two takeaways from that grid. First, Qwen2.5-Coder 7B at q4_K_M is the default for a reason — it fits comfortably, generates fast enough to feel responsive in chat, and Qwen's 2.5-generation coder models are the strongest 7B open weights available as of 2026 on the community HumanEval and BigCodeBench aggregates. Second, DeepSeek-Coder-V2 Lite in its 16B MoE form is the aspirational choice on this card — it barely fits, but its mixture-of-experts routing means only ~2.4B parameters activate per token, so tok/s is surprisingly livable and the code quality on longer generations pulls ahead of a dense 7B.

Avoid q8 on this card unless you have a specific reason. The quality delta over q5_K_M is small on coder tasks, and you are giving up almost half your tok/s for it.

Step-by-step: Ollama + Continue.dev wiring

Assume a fresh Windows 11, macOS 15, or Ubuntu 24.04 install with recent NVIDIA drivers (r550 or newer). CUDA runtime is not required — Ollama ships its own CUDA-linked llama.cpp build.

1. Install Ollama. Download the platform installer from ollama.com or use the one-liner on Linux: curl -fsSL https://ollama.com/install.sh | sh. The installer registers a systemd service on Linux and a background service on Windows/macOS. Confirm with ollama --version and curl http://localhost:11434/api/tags — an empty JSON {"models":[]} response means the server is up.

2. Pull two models. One for chat, one for autocomplete:

ollama pull qwen2.5-coder:7b
ollama pull qwen2.5-coder:1.5b-base

The 7B is your chat and edit workhorse. The 1.5B-base variant is tuned for fill-in-the-middle (FIM) completion — the tab-complete workload — and its lower parameter count keeps first-token latency below the ~200 ms threshold where autocomplete stops feeling like autocomplete.

3. Install the Continue.dev VS Code extension. Search "Continue" in the VS Code marketplace, install, reload. On first launch Continue drops a config file at ~/.continue/config.json.

4. Edit ~/.continue/config.json to point both a chat model and an autocomplete model at your local Ollama server:

{
 "models": [
 {
 "title": "Qwen2.5-Coder 7B (local)",
 "provider": "ollama",
 "model": "qwen2.5-coder:7b",
 "apiBase": "http://localhost:11434"
 }
 ],
 "tabAutocompleteModel": {
 "title": "Qwen 1.5B FIM",
 "provider": "ollama",
 "model": "qwen2.5-coder:1.5b-base",
 "apiBase": "http://localhost:11434"
 },
 "embeddingsProvider": {
 "provider": "ollama",
 "model": "nomic-embed-text"
 }
}

5. Test the round-trip. Open a code file, press Ctrl+I (Continue's inline edit shortcut), and ask it to add a docstring to the function under your cursor. If the streamed response appears within ~1 second and completes in a handful more, you're done. Autocomplete kicks in automatically as you type — a ghost-text suggestion should appear after ~150-300 ms of pause.

If autocomplete lags, drop the FIM model to qwen2.5-coder:0.5b-base or shorten the context via Continue's tabAutocompleteOptions.maxPromptTokens setting. If chat lags or thrashes VRAM, close browser tabs — VS Code's Electron process and Chrome will happily eat 2 GB of VRAM between them via WebGL/hardware acceleration and squeeze your model.

Autocomplete vs chat vs agent: which model for which job?

Not every coder-assistant task wants the same model, and matching model size to task type is what separates a setup that feels magical from one that feels like a demo.

Autocomplete is a latency-first workload. The suggestion has to appear before you finish typing the next character or it is worse than nothing. Time-to-first-token matters more than throughput, and small FIM-tuned models (0.5B-3B) shine here. Qwen2.5-Coder 1.5B-base is the current sweet spot on a 3060: fast enough that suggestions land in the 100-300 ms window, smart enough to complete a function signature or a common idiom.

Chat and inline edits are a quality-first workload. You are asking the model to write or refactor a nontrivial block, and you will read every character it produces. Throughput of 30-60 tok/s is faster than you can read, so latency is forgiving; quality matters. This is where Qwen2.5-Coder 7B q4 or DeepSeek-Coder-V2 Lite 16B q4 earn their keep.

Agent mode — where the model runs shell commands, edits multiple files, iterates on test output — is the workload where local models still lose to cloud. Agentic loops need long context, tool use, and the reasoning to notice when a plan has drifted. Cline and Continue's agent mode both work with a local 7B in principle, but community reports on r/LocalLLaMA converge on the same finding: for anything beyond a two-step tool loop, escalate to a cloud model. Do not fight this.

Benchmark table: first-token latency and throughput on the RTX 3060

Numbers below are drawn from community reports on r/LocalLLaMA and Ollama's community discussions for a stock RTX 3060 12GB (170 W TDP, 360 GB/s memory bandwidth per the TechPowerUp specs) running Ollama with default settings on a 128-token prompt at 8k context window. Your mileage varies by driver version, other GPU load, and specific quantization file.

The break-point is around 30 tok/s and 500 ms first-token latency. Below either threshold, the model starts feeling laggy for interactive chat use, and you will find yourself alt-tabbing to a cloud model instead. Above both, it feels like a tool that belongs in your workflow.

Perf-per-watt and when to fall back to cloud

An RTX 3060 pulls roughly 170 W under sustained inference — well below its 200 W board limit but well above idle. On US grid-average pricing of ~$0.16/kWh, a Ryzen 7 5700X system pulling 250 W total under load costs about $0.04/hour to run. Ten hours of active coding per week is $2/month in electricity. Against a $19/user/month Copilot Business seat, the local rig pays for itself on power alone in the first year even after amortizing the card.

But the honest comparison is not local-vs-cloud, it is local-plus-cloud. The workflow that beats either on its own is: local Qwen2.5-Coder 7B for autocomplete, inline edits, docstrings, boilerplate, and quick "explain this function" questions; cloud escalation (Claude, GPT-4o, Gemini) for cross-file refactors, novel algorithm design, debugging that requires reading a stack trace against library source, and anything where you would ask a senior engineer to pair with you. The rough allocation on a real workday is 80% local / 20% cloud, and that 20% is where the cloud spend goes.

Fall back to cloud when: the task requires more than ~16k tokens of context (local model quality degrades sharply past its training context, and the 3060 cannot hold enough KV cache for a truly long context anyway); when you need vision (screenshot debugging, UI feedback); when the local model's first draft is wrong twice in a row on the same task (a signal it is not going to converge); when you are debugging cross-repository, cross-language integration issues.

Bottom line

The RTX 3060 12GB earned an unlikely second life as the entry-level card for local AI coding assistants, and 2026's model landscape — Qwen2.5-Coder, DeepSeek-Coder-V2 Lite, StarCoder2 — finally rewards that VRAM buffer. Paired with a mid-range AM4 CPU, an NVMe drive for model storage, and Ollama plus Continue.dev, you get a self-hosted Copilot alternative that costs pennies of electricity per week, keeps your code on your machine, and is fast enough for real interactive use.

The important honesty: this is not a Copilot replacement for every workload. Local 7B coder models are excellent at autocomplete, inline edits, docstrings, and short chat questions, and they are noticeably weaker than frontier cloud models at multi-file refactors, novel algorithm work, and long-context agentic loops. The productive stance is to run local as your default and escalate to cloud when the local model's answer is not converging. That hybrid pattern is where a 3060 rig genuinely competes with a paid seat.

If you are shopping the parts list from scratch in 2026, the 3060 12GB remains the price-per-VRAM winner among current-generation NVIDIA cards; the RTX 4060 Ti 16 GB is the step-up if you want to run 13B-14B models comfortably. If you already have a 3060, you are already at the price point where the software finally caught up to the hardware.

Related guides

• Best GPUs for local LLM inference in 2026
• RTX 3060 12GB vs RTX 4060 Ti 16GB for local AI workloads
• How to pick a coder LLM: Qwen2.5-Coder vs DeepSeek-Coder-V2 vs StarCoder2
• Continue.dev vs Cline vs Aider: choosing a local-first VS Code AI extension
• Ollama tuning cheatsheet: KV cache, context length, and quantization on 12 GB cards

Citations and sources

• Ollama blog
• TechPowerUp GeForce RTX 3060 GPU specifications
• Tom's Hardware Samsung 970 EVO Plus SSD review

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.