Key Takeaways
- The official V4.1-Flash weights are 510.31 GB, about 43× one card's 12 GB, per the Hugging Face file listing.
- The KV cache is 890 bytes per token, roughly 1/4 of V4-Flash's, per the model card.
- The smallest working local quant, DwarfStar Q2, still has 151.77 GiB of main weights, per antirez's GGUF card.
- Upstream llama.cpp cannot load any V4.1 file yet. PR #28696 adds conversion only and is still open.
- Two 3060s ran a 27B model at 40-42 tok/s, falling to ~35 tok/s at 100K context, in a community field report.
- One 3060 ran a 35B MoE at 38.9 tok/s with expert offload, per InsiderLLM.
Step 0: is your bottleneck weights or KV cache?
Every local-inference memory budget has two lines, and they behave differently.
Weights are fixed. A model's quantized file has to sit in memory, split across VRAM and system RAM if necessary, before the first token is generated. Weight size depends on parameter count and bits per parameter. Context length has no effect on it.
KV cache grows linearly with context. Every token in the conversation stores key and value vectors for each attention layer. For an agent that keeps a 32K-token tool history alive, the cache can equal or exceed the weights on a dense model. Take Qwen3-14B: its config.json lists 40 layers, 8 KV heads and a head dimension of 128. That works out to 2 × 40 × 8 × 128 × 2 bytes = 163,840 bytes per token at FP16, or about 5 GiB at 32K.
The diagnostic is simple. If the model does not load at a short context, weights are your bottleneck, and no cache trick will help. If it loads but runs out of memory as the conversation grows, the KV cache is the bottleneck, and cache quantization, a shorter context or a second card will help.
DeepSeek's V4.1-Flash announcement is about the second problem. A 12 GB owner considering this model has the first problem, by a factor of more than ten. Everything below follows from that distinction.
What did DeepSeek actually change in V4.1-Flash?
Per the DeepSeek-V4.1-Flash model card and its linked technical report:
- Architecture: a multimodal MoE with 552B backbone parameters and 196B Engram parameters. It activates 8B parameters per token during prefill and 16B during decode. Each MoE layer has 1 shared expert and 384 routed experts, with 6 routed experts active per token.
- Context: up to one million tokens.
- KV cache: FP4 main KV caching (E2M1 format, one E4M3 scale per 16 channels) that "reduce[s] the global KV cache footprint to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash." The persistent cache, held on SSD or in host memory, drops to roughly 1/8 of V4-Flash's.
- Precision: FP8 for dense weights and FP4 for experts, per the repository's config.json.
- License: MIT.
The Decoder adds context from DeepSeek's materials: per-token global KV fell by a factor of 437 compared with DeepSeek V1, and the model scores 74.2 percent on DeepSWE v1.1. Its article covers API and datacenter use, not consumer hardware.
The predecessor for comparison: DeepSeek-V4-Flash has 284B parameters (13B activated). Its original repository totals 159.63 GB, and the July 0731 refresh totals 166.90 GB.
So the memory claim is accurate. It describes cache per token at long context, which matters most to datacenter operators serving many million-token sessions at once. The weights V4.1 adds more than offset that saving on a single home box.
Quantization matrix
File sizes are summed from each repository's Hugging Face file listing. The KV figures are arithmetic from the model card's 890 bytes per token: 8,192 × 890 ≈ 7 MiB and 32,768 × 890 ≈ 28 MiB. The runtime's own buffers add more on top.
| Build | Weights on disk | KV cache at 8K / 32K | Fits 12 GB? | Fits 24 GB? | Quality / status |
|---|---|---|---|---|---|
| Official FP8 + FP4 experts | 510.31 GB | ~7 MiB / ~28 MiB | No | No | Reference weights; vLLM floor 614 GB |
| Q8_0 (vcruz305) | 507.95 GB | ~7 MiB / ~28 MiB | No | No | "do not run on upstream llama.cpp yet" |
| Q4_K_M (vcruz305) | 444.74 GB | ~7 MiB / ~28 MiB | No | No | Same status |
| Q3_K_M (vcruz305) | 347.27 GB | ~7 MiB / ~28 MiB | No | No | Same status |
| Q2_K (vcruz305) | 264.52 GB | ~7 MiB / ~28 MiB | No | No | Author: "Q2_K is the floor" |
| Q1_0 (vcruz305, withdrawn) | n/a | n/a | n/a | n/a | "emitted one repeated token for every prompt" |
| DwarfStar Q2 (antirez) | 340.60 GiB total, of which 151.77 GiB main weights | ~7 MiB / ~28 MiB | No | No | Runs via SSD streaming on a 128 GB Mac or DGX Spark |
| Mixed Q2 without Engram (apetersson) | 168.93 GB | ~7 MiB / ~28 MiB | No | No | Card: does not fit one 96 GB RTX PRO 6000 |
There is no FP16 row because DeepSeek does not publish FP16 weights. The native release is already FP8 and FP4. None of the vcruz305 files load in upstream llama.cpp, so their quality loss has not been measured. The quantizer withdrew the Q1_0 build because it produced degenerate output.
The KV column is the point of the release: at 32K context, the cache is smaller than a single browser tab. The weights column is why that doesn't help a 3060 owner. Even the most aggressive build that runs is about 12.6× one card's memory and 6.3× two cards' memory, before counting the Engram tables.
Single RTX 3060 12GB: what actually fits, and where it falls over
Two street-available 12 GB cards cover this build: the MSI Gaming GeForce RTX 3060 12GB and the ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB. The NVIDIA RTX 3060 family page lists 12 GB of GDDR6 on a 192-bit memory interface, 3,584 CUDA cores, 170 W of graphics card power and 550 W of required system power. More measurements for the chip are on the RTX 3060 benchmark page.
V4.1-Flash on one card falls over before it loads. No published runtime path puts it on a GeForce card. antirez's DwarfStar runtime, the only local GGUF runtime that runs V4.1 today, documents V4.1 text and vision on Metal and text on DGX Spark. It lists no consumer-NVIDIA path.
What one 3060 runs well for agents instead, per public measurements:
| Model and quant | Setup | Generation | Prompt processing | Source |
|---|---|---|---|---|
| Llama 2 7B Q4_0 | Fully on GPU, flash attention | 76.92 tok/s | 2,407.67 tok/s (pp512) | llama.cpp #15013 |
| Qwen3 14B Q4_K | Fully on GPU, 16K context | 22.7 tok/s | 678.2 tok/s | Hardware Corner |
| Qwen3.6-35B-A3B UD-Q4_K_M | -ncmoe 24, 32 GB DDR4 | 38.9 tok/s (38.2 at 8K) | 413 tok/s | InsiderLLM |
The 35B mixture-of-experts row is the one that matters for agents. The model has 35B parameters but activates only about 3B per token, so llama.cpp can keep attention on the GPU and move expert weights to system RAM with --n-cpu-moe. The InsiderLLM run sat at 9.8 GB of VRAM. That page also warns that a 3060 driving a desktop display loses ~0.5-1 GB, which pushes -ncmoe 24 into an out-of-memory error. Raise the value to 26 or 28 on a card that runs your monitor.
Where one card falls over is dense models above ~14B at long context. The weights fit, but the FP16 KV cache from Step 0 does not.
Dual RTX 3060 24GB: layer split, PCIe lane cost, and measured scaling
The most detailed public dual-3060 report is a September 2026 field report running Qwen3.8-27B on two RTX 3060s. The host was a Xeon E5-2680 v4 with 32 GB of DDR4-2400 and both cards on PCIe 3.0 x16. The figures are self-reported.
| Configuration | Prompt processing | Generation | Context |
|---|---|---|---|
| 2× RTX 3060, llama.cpp | ~620 tok/s at start, ~450 tok/s at ~100K | 40-42 tok/s at start, ~35 tok/s at ~100K | Up to ~100K |
| 2× RTX 3060, vLLM tensor parallel (TP2) + MTP + FP8 KV | ~3,898 tok/s peak logged | 49.2-63.8 tok/s logged | Not stated |
| 1× RTX 3060, Qwen3 14B Q4_K (reference) | 678.2 tok/s | 22.7 tok/s | 16K (Hardware Corner) |
Two points stand out. First, the second card mainly buys model class. A 27B dense model with a 100K context fits across 24 GB and doesn't fit on 12 GB. Generation at that scale (40-42 tok/s) beats the single card's 14B figure (22.7 tok/s), because the card pair is running a larger model with no offload, not because layer splitting doubles speed. Second, the serving stack matters as much as the hardware. The same two cards logged 49-64 tok/s under vLLM tensor parallelism, above llama.cpp's layer split.
PCIe lane cost. The field report ran both cards at x16 on a server platform. On a consumer AM4 board, a second GPU usually drops the slots to x8/x8, or x8/x4 through the chipset. For layer-split inference, only small activations cross the bus between cards, so that has little effect. Tensor parallelism exchanges data between cards on every layer, and it is more sensitive to link width.
Power. The report's author limited both cards to 135 W from the default 170 W. That brings a two-card build's GPU budget to 270 W.
Prefill vs generation: why agent workloads punish the single-card build hardest
A chat turn is mostly generation. An agent turn is mostly prefill. Each tool call returns JSON, a web page or a file listing, and the model has to process that output before deciding what to do next. With prompt caching, only the new tokens are prefilled. Without it, or whenever a harness rewrites earlier messages, the whole history is prefilled again.
The numbers show why this hits a single card harder. With expert offload, one 3060 prefilled the 35B MoE at about 413 tok/s (InsiderLLM). At that rate, an 8,000-token tool result takes about 19 seconds before the first new token appears. The dual-card llama.cpp build held about 450-620 tok/s on a larger dense model, and vLLM on the same pair logged bursts near 3,900 tok/s (field report). Over a 30-turn agent run, the prefill difference decides whether a task takes minutes or most of an hour.
Offload is what hurts prefill most. Every prompt token has to pass through expert weights held in system RAM, and the CPU does that work. A model that fits entirely in VRAM, which two cards allow more often, avoids that cost.
Context-length impact: 8K / 32K / 64K KV budget
This table compares V4.1-Flash's cache, a dense 14B model on one card, and the same dense model with 8-bit cache. Dense-model figures are computed from Qwen3-14B's config.json at 163,840 bytes per token for FP16. q8_0 is roughly half of that.
| Context | V4.1-Flash KV (890 B/token) | Qwen3-14B KV, FP16 | Qwen3-14B KV, q8_0 | Qwen3-14B Q4 weights + FP16 KV on 12 GB? |
|---|---|---|---|---|
| 8K | ~7 MiB | ~1.25 GiB | ~0.66 GiB | Yes |
| 32K | ~28 MiB | ~5.0 GiB | ~2.7 GiB | No at FP16 (~13.4 GiB); tight with q8_0 (~11 GiB) |
| 64K | ~56 MiB | ~10.0 GiB | ~5.3 GiB | No; needs a second card or offload |
The fit column adds the 8.38 GiB Qwen3-14B Q4_K_M file from bartowski's GGUF repo, before compute buffers. Qwen3 is 32,768 tokens native and 131,072 with YaRN, per the model card, so the 64K row assumes YaRN is enabled. The takeaway still holds. On dense models, the second 12 GB card is what pays for long context, and 8-bit KV-cache quantization is the cheaper first step. llama.cpp enables it with --cache-type-k q8_0 --cache-type-v q8_0, and quantizing the V cache requires flash attention.
V4.1-Flash's cache column shows what DeepSeek solved: 64K of context in the space of a thumbnail image. Unfortunately, that saving sits behind a 510 GB set of weights.
Does the host CPU matter?
For a model that fits entirely in VRAM, the host CPU mostly queues work for the GPU. For a model with offloaded experts, the CPU and its memory bandwidth set the speed. Dual-channel DDR4-3200 moves about 51.2 GB/s (2 × 25.6 GB/s). The InsiderLLM run reached 38.9 tok/s on an older i7-7700 with DDR4-2133. The expert-offload traffic in that run is light because only ~3B parameters are active per token.
The AMD Ryzen 7 5800X is the host to buy if you plan to offload deliberately. Eight cores and 16 threads handle expert computation and the agent harness at the same time. The AMD Ryzen 5 5600G is the cheaper always-on host. Its integrated Radeon graphics can drive a display, so the 3060 keeps its full 12 GB for the model, which the InsiderLLM warning above shows is worth 0.5-1 GB.
Neither CPU changes the V4.1-Flash answer. DwarfStar Q2's 151.77 GiB of main weights exceed the 128 GB that four 32 GB DIMMs provide on a typical four-slot AM4 board. Beyond that, no runtime exists to offload V4.1 to a consumer NVIDIA card.
Model library storage
Agent builders tend to accumulate model files. A 35B MoE at Q4_K_M is about 22 GB, a 27B dense model is similar, and a 14B model is about 9 GB. The Crucial BX500 1TB SATA SSD, rated up to 540 MB/s in its listing, keeps that library off the boot drive.
Load-time arithmetic at 540 MB/s:
| File | Size | Sequential read time at 540 MB/s |
|---|---|---|
| Qwen3.6-35B-A3B UD-Q4_K_M | ~22 GB | ~41 s |
| 14B dense Q4 | ~9 GB | ~17 s |
| V4.1-Flash official weights | 510.31 GB | ~945 s (~16 min) |
For models that load once and serve for hours, a SATA drive's slower load barely matters. It matters a lot for DwarfStar-style SSD streaming, which reads Engram rows from disk during inference. At SATA speeds, that design would be starved. That is one more reason V4.1-Flash is a Mac Studio or DGX Spark model and not a 3060 model.
Perf-per-dollar and perf-per-watt
Catalog prices on SpecPicks at the time of writing are $479.99 for the MSI RTX 3060 12GB, $499.99 for the ZOTAC Twin Edge OC 12GB, $254.90 for the Ryzen 7 5800X, $199.99 for the Ryzen 5 5600G and $163.49 for the BX500 1TB. Prices change, so check the linked product pages.
| Build | GPU cost (catalog) | GPU power budget | Best measured agent-class result | What the second card buys |
|---|---|---|---|---|
| 1× RTX 3060 12GB | ~$480 | 170 W | 35B MoE at 38.9 tok/s with offload | n/a |
| 2× RTX 3060 12GB | ~$960-980 | 340 W at default, 270 W at 135 W cap | 27B dense at 40-42 tok/s, ~35 tok/s at 100K | Dense 27B-class models, ~100K context, vLLM TP2 up to ~64 tok/s |
| Either build, V4.1-Flash | n/a | n/a | Does not run | Nothing |
The second card roughly doubles GPU cost and power. It does not double tok/s on the same model. What it buys is the ability to run a dense 27B-class model with a long context and no offload. If your agent does well on a sparse MoE, the single card is the better buy per dollar. If it needs a dense mid-size model with a 100K history, the second card is what makes that possible.
Verdict matrix
Get one 3060 if… your agent runs well on a sparse MoE such as a 35B-A3B model with expert offload. Context stays under ~32K, and you have 32 GB of system RAM.
Get two 3060s if… you need a dense 27B-class model resident in VRAM with a context approaching 100K, or you want to try vLLM tensor parallelism for faster prefill on long tool histories.
Skip both and rent (or use the API) if… DeepSeek V4.1-Flash specifically is the model you need. Its weights need hundreds of gigabytes, vLLM's recipe starts at 614 GB of GPU memory, and DeepSeek serves it through its own API.
Recommended pick
For a local agent rig in September 2026, the dual RTX 3060 build wins, but not for V4.1-Flash. Two MSI RTX 3060 12GB cards give 24 GB. The measured field report shows a 27B-class model holding ~35 tok/s at 100K of context, which covers the long tool histories agents build up, and vLLM on the same pair logged up to ~64 tok/s. Buyers who don't need dense models or long context should stop at one card and run a 35B MoE with offload. Anyone set on V4.1-Flash should use DeepSeek's API.
Bottom line
DeepSeek V4.1-Flash reduced the KV cache and increased the weights, and the weights decide whether a model fits on a 3060. Neither one card nor two can load it. Two cards remain the better 24 GB agent rig for dense models, and one card with a sparse MoE is the value choice.
Related guides
- Best parts for a dual RTX 3060 24GB local LLM build
- RTX 3060 benchmarks
- The Q4_K_M agent trap: KV-cache quant math
- DeepSeek V4-Flash on an RTX 3060 12GB
Live price comparison
Compare current pricing for the two 12 GB cards on the MSI Gaming GeForce RTX 3060 12GB and ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB product pages. The host options are the AMD Ryzen 7 5800X and the AMD Ryzen 5 5600G, and the library drive is the Crucial BX500 1TB. Prices may vary.
Citations and sources
All sources accessed 2026-09-17.
- DeepSeek-V4.1-Flash on Hugging Face: parameters, experts, 1M context, 890 bytes/token KV cache, MIT license and file sizes.
- DeepSeek-V4.1-Flash config.json: FP8 dense and FP4 expert precision.
- DeepSeek API news, 2026-09-10: release announcement.
- vLLM Recipes: DeepSeek-V4.1-Flash: 614 GB minimum GPU memory with 1.2× headroom.
- The Decoder: V4.1-Flash cuts memory needs for AI agents: release coverage, 437× KV reduction versus V1, DeepSWE v1.1 score.
- DeepSeek-V4-Flash on Hugging Face: predecessor's 284B / 13B-active specification.
- vcruz305/DeepSeek-V4.1-Flash-GGUF: Q2_K to Q8_0 sizes, the withdrawn Q1_0 and upstream llama.cpp status.
- antirez/deepseek-v4.1-flash-gguf: DwarfStar Q2 sizes and SSD streaming requirements.
- apetersson/DeepSeek-V4.1-Flash-MixedQ2-GGUF: mixed Q2 without Engram, and the RTX PRO 6000 fit result.
- llama.cpp PR #28696: converter-only V4.1 support, open.
- antirez/ds4 (DwarfStar): supported V4.1 backends (Metal, DGX Spark).
- NVIDIA GeForce RTX 3060 family page: 12 GB GDDR6, 192-bit interface, 170 W, 550 W system power.
- llama.cpp discussion #15013: RTX 3060 12GB Llama 2 7B Q4_0 scoreboard row (with flash attention).
- Hardware Corner RTX 3060 12GB LLM benchmarks: Qwen3 14B Q4_K at 16K context.
- InsiderLLM: Qwen3.6-35B MoE locally: single-3060 expert-offload measurements.
- Field report: 2× RTX 3060 24 GB: dual-card llama.cpp and vLLM throughput and power limits.
- Qwen3-14B config.json: layer and KV-head counts used for the dense-model KV math.
- bartowski/Qwen_Qwen3-14B-GGUF: Q4_K_M file size.
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
