Best Local LLM for Coding Agents on a 24GB GPU (Late 2026)

As of October 2026, the best local model for running a coding agent on a single 24GB GPU is Qwen3.8-27B at 4-bit. Its Q4_K_M GGUF is 16.46 GB on Hugging Face, and Hardware Corner measured 45.5 tok/s generation on an RTX 4090 and 39.8 tok/s on an RTX 3090 at 8k context (Q4_K_S, llama.cpp, MTP off). Turning on the model's built-in multi-token-prediction (MTP) head in llama.cpp raises that substantially: a preregistered RTX 3090 study archived in September 2026 measured 66.39 tok/s with --spec-draft-n-max 2, up from 41.55 tok/s without it (+59.8%, UD-Q4_K_XL, 8k context). The same study notes that MTP output is not bit-exact with ordinary step-by-step greedy decoding. Qwen's model card lists 61.7 on SWE-bench Pro, up from 53.5 for Qwen3.6-27B. If you care more about decode speed than peak quality, look at GLM-4.7-Flash, a 30B-A3B MoE that generated 93 tok/s on an RTX 3090 at 4k context. If you want a dense model with a small memory footprint, look at Devstral Small 2 24B, whose Q4_K_M file is 14.33 GB.

What changed since September 2026: Off llama.cpp, the HyperQwen patched-vLLM project reports 127 tok/s single-stream for Qwen3.8-27B on one RTX 3090 at 250 W (W4A16, 64k context, project-reported), and RTX 3090 eBay listings averaged a $1,373 lowest-average price in October 2026.

What changed — October 2026

• MTP speculative decoding is now the biggest speed lever for Qwen3.8-27B on 24GB. Per a preregistered llama.cpp study (Zenodo v1.0.4, deposited 2026-09-02), a single RTX 3090 decoded at 66.39 tok/s with --spec-draft-n-max 2, against a 41.55 tok/s baseline (+59.8%), using the 17.56 GB UD-Q4_K_XL GGUF at 8k context. The study reports that MTP output is not bit-exact with ordinary step-by-step greedy decoding.
• Agent-style tool-call workloads gain even more from MTP, per one community report. The sudoingX/qwen38-mtp benchmark table lists an RTX 3090 JSON/tool-call run dated 2026-09-26 (UD-Q4_K_XL, 65,536 context) at 34.0 tok/s without MTP and 84.4 tok/s at n-max 4, with 0.971 draft acceptance. The table labels it one synthetic prompt, not an application-speed or quality claim.
• A patched vLLM stack roughly triples single-stream decode, per its own repo. The HyperQwen README lists 127 tok/s single-stream in single-user mode at 64k context, and about 1,035 tok/s aggregate decode at 64 concurrent requests in batch mode, on one RTX 3090 at 250 W. That compares with ~40 tok/s for plain llama.cpp on the same card. Its quality page puts the W4A16 single-user stack at 78.3 IFBench prompt-level strict, against 79.5 on Qwen's card, about one point lower. The repo states every figure comes from one card shared with training jobs, so treat these as project-reported numbers until others reproduce them.
• RTX 3090 eBay pricing has risen. Per GPUPoet's eBay tracker, the RTX 3090's lowest-average price (the mean of the three cheapest listings each day) was $1,373 in October 2026, with a daily range of $1,325 to $1,437, and the tracker reports its lowest-average price up 15.7% over the four months it charts.

What changed — September 2026 refresh

• New default pick. Qwen3.8-27B replaces the April recommendation, Qwen 3.6 27B. Hardware Corner measured it at 45.5 tok/s on an RTX 4090 and 39.8 tok/s on an RTX 3090 at 8k context. Qwen's model card lists 61.7 on SWE-bench Pro, up from 53.5 for Qwen3.6-27B.
• Three current models added with measured speeds. The three are GLM-4.7-Flash at 93.4 tok/s at 4k context on an RTX 3090, Devstral Small 2 24B at 44 tok/s at 16k, and Gemma 4 26B-A4B at 128 tok/s on an RTX 3090. Each row lists its real Hugging Face GGUF size and a bandwidth ceiling.
• Models that do not exist were removed. "DeepSeek-Coder V2.5 16B-A2.4B", "GLM-4 32B-A9B", "Llama 3.3 33B" and "DeepSeek-Coder V3 32B-A8B" are not released open-weight models, so they were dropped.
• Uncited internal-eval tables were replaced. The tool-call accuracy, file-edit accuracy, per-quant HumanEval+/KLD and SWE-bench-subset tables had no public source. In their place are vendor model-card scores, measured context-scaling and prefill tables, and Quesma's August 2026 quantization benchmark.
• The sources section was rebuilt with one entry per inline URL.

What's changed as of September 2026

This guide first ran in April 2026 and recommended Qwen 3.6 27B. Three things have changed since then:

• Qwen3.8-27B shipped in August 2026 and replaces Qwen3.6-27B as the default. It has the same 27B dense footprint and scores higher on every coding benchmark its model card reports: Terminal Bench 2.1 rose from 63.4 to 73.0, SWE-bench Pro from 53.5 to 61.7, and LiveCodeBench v6 from 83.9 to 90.3. Qwen ran the same evaluation harness for both models.
• Coding-specialist MoE models now fit comfortably on 24GB. GLM-4.7-Flash (January 2026) and Gemma 4 26B-A4B both have Q4 GGUFs of about 17–18 GB. They activate only 3–4B parameters per token, so they decode 2–3× faster than a dense 27B.
• Several models named in the April version have been removed. "DeepSeek-Coder V2.5 16B-A2.4B", "GLM-4 32B-A9B", "Llama 3.3 33B" and "DeepSeek-Coder V3 32B-A8B" do not exist as released open-weight models on Hugging Face. The internal tool-call and file-edit tables built on them have been removed too. Every figure below links to a public model card or a published measurement.

Current-generation models that fit 24GB, with measured speeds

How to read the ceiling column: decode is memory-bandwidth-bound, so a card can't generate faster than its bandwidth divided by the bytes read per token. The RTX 3090 has 936 GB/s and the RTX 4090 has 1,008 GB/s.

• Dense models read the whole weight file for every token.
• MoE models read only the active experts. For those rows the file size is scaled by active ÷ total parameters: 3B of 31.2B for GLM-4.7-Flash, and 3.8B of 25.2B for Gemma 4 26B-A4B.

Every measured figure above sits below its ceiling. The MoE rows land far below theirs because routing overhead and attention costs dominate at that size. File sizes come from each repo's Hugging Face file listing. Hardware Corner reports only "Q4_K" for Devstral, so the ceiling above assumes the larger Q4_K_M file, which is the conservative case.

Why coding agents break local models in ways chat evals don't capture

A model that does well on single-turn code benchmarks can still produce a malformed tool-call argument every few dozen turns. When that happens it kills the agent loop and wastes an hour. Chat evals don't measure this, and single-issue benchmarks only partly do. To really measure it, run the model inside an actual agent harness (Aider, Continue, Roo, OpenCode, Claude-Code-style CLIs, or your own) on a task that takes 20+ tool calls. Then count how often the harness has to retry because the model:

• emitted malformed JSON,
• hallucinated a function signature, or
• forgot the contents of a file it edited four turns ago.

That is why the 2026 model cards have shifted toward agentic benchmarks. Qwen, for example, reports SWE-bench Pro and DeepSWE evaluated inside the Claude Code harness at a 256K context window, rather than leading with HumanEval-style pass@1.

This article covers the decision a 24GB GPU buyer actually faces: which open-weight model, at which quant, gives the highest sustained agent throughput before the loop derails. The criteria are:

• Agentic benchmark scores, as reported by the model vendors.
• Measured decode speed, from public benchmarks.
• Prefill speed. Agent loops resend 8–16k tokens of context every turn, so you often spend more time prefilling than generating.

Key takeaways

• Qwen3.8-27B at Q4 is the default pick for a general-purpose coding agent on 24GB. It has the strongest agentic scores of any model that fits: 73.0 on Terminal Bench 2.1 and 61.7 on SWE-bench Pro. It still decodes at ~40–46 tok/s on a 3090/4090.
• Q4_K_M costs nothing measurable on agentic coding. In an independent August 2026 test, the 17 GB Q4_K_M matched the full BF16 model on Terminal-Bench 2.1. Scores dropped only at 2-bit and collapsed at 1-bit.
• GLM-4.7-Flash is the speed pick. It is a 30B-A3B MoE with 59.2 on SWE-bench Verified and 93 tok/s at 4k context on a 3090. It slows sharply at long context.
• Devstral Small 2 24B is the small-footprint dense pick. Mistral reports 68.0% on SWE-bench Verified, and it runs at 44 tok/s at 16k context on a 3090.
• Prefill matters more than generation speed for agent loops. An RTX 4090 prefills Qwen3.8-27B at ~2,850 tok/s at 8k context versus ~1,235 tok/s on an RTX 3090. That gap is much wider than the 15% gap in generation speed.

What actually breaks when a coding agent runs on a small local model?

Three failure modes cause most agent derailments. None of them show up in single-turn benchmarks:

1. Tool-call schema drift. The model is supposed to emit a JSON object that matches a tool schema. Smaller models sometimes emit a trailing comma, an unescaped backslash inside a regex, or a smart-quote character instead. Most agent harnesses retry, but each retry means another full prefill of the conversation. Three retries in a row will convince the user the agent has frozen.

2. Cross-turn memory drift. The agent reads file A in turn 3 and edits it in turn 5. Then in turn 9 the model proposes an edit based on the original contents of file A. The diff from turn 5 has fallen out of the model's effective attention, even though it is still in the literal context. Effective context length and advertised context length are not the same number. This is one reason the 2026 hybrid-attention designs matter. Qwen3.8-27B mixes Gated DeltaNet and gated-attention layers and advertises 262,144 tokens natively.

3. Hallucinated affordances. The model emits a tool call with a parameter the schema doesn't define, or calls a tool that doesn't exist. A bigger context window doesn't fix this. More capable models do.

A coding agent makes 20–50 tool calls per task. With a 3% tool-call failure rate, you'll see roughly one failure per task. At 1%, you'll see one every two to three tasks. That arithmetic is why a few points on an agentic benchmark matter more here than they do for chat.

Which models fit at usable quants on 24GB?

"Usable" here means the weights, plus the KV cache for at least 32k context, plus activation buffers, all fit with no offload to system RAM. After CUDA overhead, a 24GB card has roughly 21–22 GB of working set. For scale, Quesma measured an F16 KV cache for Qwen3.8-27B at about 2.3 GB per 32k tokens, which leaves room for about 64k tokens beside the 17 GB Q4_K_M. File sizes below are from Hugging Face file listings:

• Qwen3.8-27B at UD-Q4_K_M (16.46 GB) fits with 32k–64k context. Hardware Corner ran it to 64k context on both a 3090 and a 4090.
• Qwen3.6-27B at Q4_K_M (16.82 GB) has the same footprint, and is now superseded by Qwen3.8-27B.
• Devstral Small 2 24B at Q4_K_M (14.33 GB) has the most headroom of the dense options.
• GLM-4.7-Flash at UD-Q4_K_XL (17.52 GB) runs 32k context comfortably. Hardware Corner measured it out to 65k context on a 3090.
• Gemma 4 26B-A4B at UD-Q4_K_XL (17.01 GB) fits with room for context.

What doesn't fit well on 24GB:

• Any 70B model at any usable quant.
• Qwen3.6-35B-A3B at 4-bit. Its UD-Q4_K_M is 22.13 GB and its MXFP4_MOE is 21.71 GB. Once you add KV cache that exceeds the working set, so you'd need a 3-bit quant or partial offload.

Coding benchmark scores (vendor model cards)

Read across rows with care. Each vendor uses its own harness, scaffold and settings. Qwen runs SWE-bench Pro inside the Claude Code harness. Mistral's and Z.ai's SWE-bench Verified numbers use their own setups. Terminal Bench 2 and 2.1 are different versions. The cleanest like-for-like comparison is Qwen3.8-27B against Qwen3.6-27B, because both columns come from the same evaluation. Across different vendors, treat a gap of a few points as noise.

Long-context behavior: how speed holds up from 4k to 64k

The April version of this guide had a "file-edit accuracy at 8k/32k/64k" table. It came from an unpublished internal eval, so it has been removed. What public measurements do show is how decode speed degrades as context grows, and the architectures differ sharply here.

(Generation tok/s at each context length, llama.cpp, Q4 GGUFs.)

From 4k to 64k context, Qwen3.8-27B loses only about 16–17% of its decode speed, which fits its hybrid linear-attention design. GLM-4.7-Flash starts more than twice as fast but loses about 70% by 65k. By 32k it is roughly level with the dense Qwen on the same RTX 3090. If your agent routinely loads several files per turn and sits at 32k+ context, GLM's headline speed advantage mostly disappears.

Prefill speed matters more than generation for agent loops — here's why

In chat, you type a prompt of a few hundred tokens and the model generates a few hundred tokens back, so prefill is invisible. An agent loop is different. Every turn carries 8–16k tokens of conversation history, tool output and file contents, and the model emits only a few hundred tokens of tool call or diff. Most of the wall-clock time goes to prefill, so halving prefill time nearly halves agent latency.

Public prefill measurements (llama.cpp, Q4 GGUFs):

Prefill is compute-bound, while decode is bandwidth-bound. That is why the RTX 4090 prefills Qwen3.8-27B about 2.3× faster than the 3090 but decodes only about 15% faster. For agent work, the 4090's advantage over a used 3090 is bigger than generation-speed charts suggest. Price the 3090 side with current data: GPUPoet's eBay-listing tracker put its October 2026 lowest-average price at $1,373 (prices vary by listing). On a 3090, keeping prompts cache-friendly matters more: reuse the system prompt and avoid reshuffling file order between turns.

Quantization for coding workloads: where quality actually drops

Quantization quality on coding tasks doesn't always track perplexity or KL-divergence. What matters is whether the model still solves tasks. The most useful current data point is Quesma's August 2026 benchmark of Unsloth's Qwen3.8-27B GGUFs on llama.cpp, which ran Terminal-Bench 2.1, GPQA Diamond and IFBench:

• BF16 (55 GB) reproduced Qwen's reported Terminal-Bench 2.1 result.
• Q4_K_M (17 GB) matched BF16 on Terminal-Bench 2.1, and showed "little difference" on GPQA Diamond.
• UD-Q2_K_XL (10.7 GB) showed "a noticeable fall" on Terminal-Bench 2.1, and wrote about a quarter more tokens on the tasks it did solve.
• 1-bit quants (UD-IQ1_S, 6.2 GB) performed "around random chance" on GPQA Diamond.

The takeaway: on a 24GB card, Q4_K_M or an equivalent 4-bit quant (UD-Q4_K_XL, Q4_K_S) is the right choice for agent work. You lose nothing measurable, and you keep room for 32k–64k of context. Going to Q5/Q6 eats context headroom for no demonstrated benefit on these benchmarks. Going below 4-bit is only worth it if it's the difference between fitting and not fitting.

ExLlamaV2/ExLlamaV3 exl-format quants and vLLM with 4-bit formats are alternatives to llama.cpp. They trade a more involved setup for different speed profiles. Nearly all the measurements in this article are llama.cpp GGUF results, so compare against those. The exception is the HyperQwen patched-vLLM figure: 127 tok/s single-stream on an RTX 3090 with a W4A16 requantization and speculative decoding. That is a project-reported result from a different engine, so don't set it beside the llama.cpp rows as if it were like-for-like.

Spec-delta table

Verdict matrix

Pick Qwen3.8-27B at Q4 if you want the safest default for a general-purpose coding agent on 24GB. It posts the strongest agentic scores of anything that fits, it holds its decode speed out to 64k context, and 4-bit quantization didn't cost it anything measurable on Terminal-Bench 2.1. If you're not sure, pick this. Note that Qwen3.8 runs in thinking mode by default. Budget output tokens accordingly, and check that your harness handles <think> blocks.

Pick GLM-4.7-Flash if your agent does many short turns at modest context, such as quick edit-test loops or single-file tasks. There, its ~80–93 tok/s at 4–8k on a 3090 makes the loop feel instant. Z.ai recommends turning on its "Preserved Thinking" mode for multi-turn agentic tasks, per the model card. The trade-off is steep slowdown past 16k context.

Pick Devstral Small 2 24B if you want a dense model built specifically for agentic coding with the most VRAM left over. At 14.33 GB it leaves the most room for long context or a second small model. Mistral pairs it with its own Mistral Vibe CLI.

Consider Gemma 4 26B-A4B if raw decode speed on an RTX 3090 is the priority. It was measured at 128 tok/s on a 3090, against 41 tok/s for Qwen3.6-27B with the same build and prompts. Google's card does not report SWE-bench-style agentic scores, so validate it inside your harness first.

Skip Codestral 22B and Qwen3.6-27B for new setups. Both still run fine. But Codestral dates from 2024 and has been superseded by Devstral within Mistral's own lineup, and Qwen3.6-27B has been superseded by Qwen3.8-27B at the same footprint.

Bottom line

For a single 24GB GPU running a coding agent in late 2026, Qwen3.8-27B at Q4_K_M is the right default. GLM-4.7-Flash wins on short-context speed, and Devstral Small 2 wins on footprint. Choose based on the workload your agent actually runs, not on a chat-eval leaderboard. Quantize to 4-bit, no lower unless you have to: public testing shows Q4_K_M matching BF16 on agentic coding, with a visible drop at 2-bit.

If you can stretch to 32GB of VRAM (RTX 5090 territory), the picture changes. Higher-precision quants and larger MoE models such as Qwen3.6-35B-A3B at 4-bit fit with room to spare, and 64k+ context stops being a constraint. On the 24GB sweet spot, though, the models above cover every reasonable workload, and choosing between them is a matter of agent style, not raw quality.

Related guides

• Gemma 4 and Larger Qwen 3.6: What Hardware You'll Actually Need
• Best 24GB GPU for local LLM inference in 2026
• llama.cpp vs ExLlamaV2 vs vLLM — which inference engine for which workload
• How to set up Aider with a local model on a single GPU
• Quantization guide — q4_K_M vs q5_K_M vs exl2 4.65bpw

Citations and sources

• Hardware Corner, Qwen3.8 27B hardware tests (Aug 17, 2026). RTX 3090/4090 prefill and generation, llama.cpp, MTP disabled: https://www.hardware-corner.net/qwen3-8-27b-hardware-tests/
• Hardware Corner, Devstral 2 hardware requirements (Jan 16, 2026). Devstral Small 2 24B on RTX 3090: https://www.hardware-corner.net/devstral-2-hardware-requirements/
• Hardware Corner, GLM-4.7 Flash hardware tests (Feb 4, 2026). RTX 3090, UD-Q4_K_XL: https://www.hardware-corner.net/glm-4-7-flash-llm-hardware/
• InsiderLLM, "Gemma 4 Just Made My 3090 Three Times Faster" (May 11, 2026). Gemma 4 26B-A4B vs Qwen3.6-27B on RTX 3090: https://buttondown.com/insiderllm/archive/gemma-4-just-made-my-3090-three-times-faster/
• Quesma, "Benchmarking Qwen3.8 27B quantizations" (Aug 26, 2026). Terminal-Bench 2.1 / GPQA / IFBench by quant: https://quesma.com/blog/qwen38-27b-quantizations-benchmarked/
• Qwen3.8-27B model card. Coding and agentic scores, Qwen3.6-27B comparison, context length, thinking mode: https://huggingface.co/Qwen/Qwen3.8-27B
• Devstral Small 2 24B model card (Mistral). SWE-bench Verified, Terminal Bench 2: https://huggingface.co/mistralai/Devstral-Small-2-24B-Instruct-2512
• GLM-4.7-Flash model card (Z.ai). SWE-bench Verified, LCB v6, Preserved Thinking: https://huggingface.co/zai-org/GLM-4.7-Flash
• Gemma 4 26B-A4B model card (Google). Parameter counts, LiveCodeBench v6: https://huggingface.co/google/gemma-4-26B-A4B-it
• Unsloth Qwen3.8-27B GGUF file listing: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
• Unsloth Qwen3.6-27B GGUF file listing: https://huggingface.co/unsloth/Qwen3.6-27B-GGUF
• Unsloth Devstral Small 2 GGUF file listing: https://huggingface.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
• Unsloth GLM-4.7-Flash GGUF file listing: https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF
• Unsloth Gemma 4 26B-A4B GGUF file listing: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF
• Unsloth Qwen3.6-35B-A3B GGUF file listing: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
• thc1006/qwen3.8-speculative-decoding-rtx3090 — preregistered llama.cpp MTP study on one RTX 3090 (Zenodo v1.0.4, 2026-09-02) — 41.55 tok/s baseline vs 66.39 tok/s with draft-mtp n-max 2 (+59.8%), UD-Q4_K_XL, 8k context; output not bit-exact.
• sudoingX/qwen38-mtp — community Qwen3.8-27B MTP benchmark table — RTX 3090 JSON/tool-call run (2026-09-26): 34.0 tok/s without MTP vs 84.4 tok/s at n-max 4, 65,536 context.
• syv-ai/HyperQwen — patched vLLM serving stack for Qwen3.8-27B on one 24GB GPU (repo updated Oct 2026) — 127 tok/s single-stream (single-user mode, 64k context) and ~1,035 tok/s aggregate at 64 concurrent (batch mode), RTX 3090 at 250 W, project-reported.
• HyperQwen quality documentation — IFBench 78.3 prompt-level strict for the W4A16 stack vs 79.5 on Qwen's card; single-user mode is W4A16.
• GPUPoet, NVIDIA GeForce RTX 3090 price in October 2026 — eBay-listing lowest-average price of $1,373 for October 2026 (range $1,325–$1,437).

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.