GeForce Tensor-Throttle Unlock: RTX 3060 12GB vs RTX 4090 for Local LLMs

Introduction

In late October 2026, Tom's Hardware reported that a firmware exploit removes a half-speed AI throttle on GeForce RTX 20, 30 and 40-series cards. Its headline claims it "doubles RTX 4090's peak tensor rate" and lifts RTX 3090 Ti matrix math "up to 80%". Those two figures belong to that report and the community projects behind it. Nothing below re-measures them.

The throttle is not a secret. NVIDIA's own whitepapers list two FP16 tensor rates for every GeForce card: one with FP16 accumulation and one with FP32 accumulation, and on GeForce the second is half the first. The workstation and datacenter parts built on the same silicon run both at full speed, and that gap is part of how NVIDIA segments its product lines. The rate is set by a speed-select register that the normal driver cannot write.

Per Hardware Busters' write-up, the Ada tool is an open-source Linux kernel module, gpu-unlock-ada. It runs a firmware transaction that changes the speed-select setting, resets the card and hands it back to the stock NVIDIA driver. Ampere support comes from the upstream Zagitta/gpu-unlock project. Neither one flashes the VBIOS, and both are Linux-only for now.

This synthesis is for people who already run local LLMs on an RTX 3060 12GB or an RTX 4090, or who are about to buy one, and want to know whether the hack changes their tokens per second. The short answer is that it depends on how long your prompts are and which runtime you use, and for the most common setup (llama.cpp with a quantized GGUF) it may change nothing at all.

Key Takeaways

• Prefill is compute-bound; generation is bandwidth-bound. The unlock raises one compute rate. It does not add a single GB/s of memory bandwidth, and bandwidth sets the generation ceiling.
• llama.cpp mostly sidesteps the throttled path already. Per the llama.cpp build docs, quantized models use its MMQ kernels by default on GPUs with INT8 tensor cores, and cuBLAS runs with "speed-optimized compute types". The unlock does not touch INT8 rates.
• The published gains are issue rates, not app speedups. The gpu-unlock-ada README calls its figures "register-resident issue rates, not application GEMM or inference speedups". One owner's cuBLAS FP16 GEMM gained 12-34%, not 100%.
• It is not risk-free. The module must load before anything else touches the GPU, multi-GPU setups are unvalidated, and NVIDIA could close the hole in a later driver or firmware revision.
• Nothing in the report measures LLM tok/s on a 3060. The one published LLM serving test on a modified 4090 found no gain (0.983× at 8K prefill).

What exactly does NVIDIA throttle on GeForce cards?

The rows below come from NVIDIA's whitepapers where NVIDIA publishes the figure, and are derived from tensor-core count × boost clock where it does not. Dense rates only (no sparsity).

Sources: RTX 3090 from Table 9 of NVIDIA's Ampere GA102 whitepaper; RTX 4090 from Appendix A of the Ada whitepaper. The RTX 3060 12GB and RTX 4070 are not in either whitepaper. Their rows use the whitepapers' per-tensor-core rate (256 FP16 FLOPs per tensor core per clock, the rate implied by the 3090 and 4090 rows) multiplied by the tensor-core counts and boost clocks on TechPowerUp's RTX 3060 12GB spec page and RTX 4070 page (112 cores at 1,777 MHz; 184 cores at 2,475 MHz). Treat them as estimates.

Two things stand out. First, the same Ampere whitepaper lists the older TITAN RTX at 130.5 TFLOPS with either accumulator, so the half rate on GeForce is a product choice, not a silicon limit. Second, the bandwidth column spans almost 3× between the 3060 and the 4090, while the throttled compute column spans more than 6×. That difference matters below.

Why does generation speed barely move?

During generation, a dense model produces one token per forward pass, and each pass has to read every weight from VRAM. So the tok/s ceiling is roughly memory bandwidth divided by the bytes of weights read per token. Tensor math per token is small by comparison; the cores spend most of their time waiting on memory.

Worked example using the model the llama.cpp scoreboard standardizes on, Llama 2 7B at Q4_0 (llama-bench reports it as 3.56 GiB, about 3.8 GB). For an 8B model at Q4_K_M (about 4.9 GB), scale each ceiling down by roughly a fifth.

Generation figures: llama.cpp CUDA scoreboard, tg128, flash attention on. Ceilings are this synthesis's arithmetic and ignore the KV cache and activations, which also have to be read.

Neither card hits its theoretical ceiling, but the gap is mostly kernel-launch overhead, synchronization and KV-cache reads, not tensor math. At batch size 1, generation multiplies a vector by each weight matrix, and llama.cpp runs those quantized matrix-vector products in its own dequantize-and-dot kernels rather than in FP16 tensor-core GEMMs. Doubling the FP32-accumulate tensor rate cannot push either card past a memory limit. At best it trims a small slice of each token's time. Expect low single digits, if anything, and no one has published a measured number for it yet.

Where would the unlock show up: prefill and long-context ingestion?

Prefill is different. The whole prompt goes through the model in large batches, so the weights are read once per batch and reused across hundreds of tokens. That turns the workload into a large matrix multiply, and big matrix multiplies are compute-bound. This is where a doubled tensor rate could appear.

Published llama-bench numbers from the same scoreboard, one build per card:

Prefill on the 4090 is 6.1× the 3060's, close to the 6.5× gap in throttled tensor rate. Generation is 2.5×, close to the 2.8× bandwidth gap. The two phases track two different specs, which is the whole argument in one table.

Across every published run on the SpecPicks benchmark pages, the RTX 4090 generates at a median of about 128 tok/s on 7-9B models and about 38 tok/s on 27-35B models, and the RTX 3060 12GB at a median near 60 tok/s on 7-9B models. Those medians mix runtimes and quants, so treat them as a range rather than a spec.

How much prefill gains depends on the runtime. Hardware Busters reports prefill "reportedly picks up 14 to 36%" with the unlock, and the gpu-unlock-ada README cites one owner's cuBLAS FP16 GEMM gaining 12-34% at two matrix shapes and nothing at 4096³. Neither is a llama.cpp GGUF measurement.

Which runtimes even use FP32 accumulation?

The unlock only helps if your runtime sends work down the FP16/BF16-with-FP32-accumulate path. Here is how the common stacks look, described conditionally because no runtime-by-runtime measurement of the unlock exists yet:

• llama.cpp (CUDA). Per its build docs, "MMQ kernels are enabled by default on GPUs with int8 tensor core support". Every RTX 20/30/40 card has those, so quantized GGUF prefill runs on INT8 tensor math, which the gpu-unlock-ada README says did not change. The cuBLAS fallback uses speed-optimized compute types by default. An FP16 or BF16 GGUF, or a build forced to cuBLAS with FP32 compute, is where a gain is plausible. Likely effect for typical Q4 users: little to none.
• vLLM. vLLM serves FP16/BF16 weights through PyTorch and custom kernels, and PyTorch's CUDA GEMMs accumulate in FP32 by default per its CUDA semantics notes. Batched serving of unquantized models is the most likely LLM winner. Quantized formats (AWQ, GPTQ, FP8) vary by kernel; the one published serving test, a quantized MoE on a modified 4090, saw 0.983×.
• ExLlamaV2. ExLlamaV2 uses its own CUDA kernels for EXL2 quants. Whether its prefill GEMMs hit the throttled path varies by kernel and build, and nobody has published unlocked numbers.
• Image generation (ComfyUI, SDXL, Flux). Diffusion models run BF16/FP16 convolutions and attention through PyTorch, which is compute-bound. Per Hardware Busters, a ComfyUI BF16 job on an RTX 3090 Ti dropped from 72 to 45 seconds. That is the strongest application-level result so far, and it is not an LLM.
• LoRA fine-tuning. Training is compute-heavy and runs in BF16 with FP32 accumulation in PyTorch, so it is a plausible beneficiary. Varies by trainer; no published measurement.

Does context length change the picture?

Yes. The longer the prompt, the bigger prefill's share of total wait time, and the more a prefill speedup is worth. Hardware Corner's RTX 3060 12GB benchmarks list Qwen3 8B at three context sizes. Using those rates and a 500-token answer:

Times are this synthesis's arithmetic (prompt tokens ÷ prefill rate, 500 ÷ generation rate) on the cited rates. Real runs vary with batch size and flash-attention settings.

Now apply the upper end of the reported prefill gain, 36%, and assume the runtime actually uses the throttled path. A 4K chat turn drops from about 11.5 s to 10.9 s, a 6% saving you would struggle to feel. A 32K RAG query drops from about 58.6 s to 47.2 s, roughly 19% faster. At 128K context, prefill would dominate even more, but no published RTX 3060 12GB run at 128K exists: the KV cache for an 8B model at that length does not fit beside the weights in 12 GB without heavy cache quantization. On a 4090 the same logic holds with every number scaled up.

So the people who could notice the unlock are coding-agent and RAG users who paste whole repositories or long PDFs into every request. Chat users mostly cannot.

Is it worth flashing?

Strictly, there is nothing to flash. Per the gpu-unlock-ada README, it is "not a VBIOS flash": a kernel module runs the unlock at boot, before the NVIDIA driver binds the card, and a stock boot entry undoes it. That lowers the brick risk compared with a ROM flash, but the risks are real:

1. Warranty and terms. Per the gpu-unlock-ada README, the method depends on a published payload and decrypted debug-booter analysis, and NVIDIA firmware stays under NVIDIA's terms, and a board maker would be within its rights to refuse a claim on a card used this way.
2. Boot-order fragility. The module refuses to run on a card that NVIDIA's driver has already bound. Loading nvidia-smi, CUDA or NVML first, or a kernel upgrade that changes the boot entry, means no unlock. The README advises a full power-off for the first attempt.
3. Driver and firmware updates. Per Hardware Busters, NVIDIA could close the bypass in a later driver or firmware revision. Pinning a driver to keep the unlock means skipping fixes.
4. Stability and heat. The Zagitta FAQ says damage is "very unlikely" and that the card's thermal and power limits stay in force, but it may run hotter doing more compute. One unlocked 4090 drew 346 W peak versus 313 W stock in the gpu-unlock-ada notes. Long-term stability testing has not been published.
5. Unverified hardware. Results come from one or a few cards per device ID. GA106 (RTX 3060) is on the supported list upstream, but no published RTX 3060 12GB result exists. Multi-GPU rigs and the display card itself are unvalidated.

It's the right call when you batch long prompts every day on a Linux box you can afford to troubleshoot, ideally a secondary card out of warranty. It is not worth it for chat-length prompts, for Windows-only setups, or on the only GPU you own.

Perf-per-dollar and perf-per-watt: RTX 3060 12GB vs RTX 4090

Using launch MSRP ($329 for the RTX 3060 12GB, $1,599 for the RTX 4090, per NVIDIA's launch pricing) and rated board power (170 W vs 450 W), with the stock scoreboard numbers from above:

Arithmetic on the llama.cpp CUDA scoreboard Llama 2 7B Q4_0 figures. Rated board power is a ceiling, not measured draw. Street prices differ from MSRP; the live price buttons on each product card carry current pricing.

The 4090 already wins prefill value at stock, and any real unlock gain widens that lead. The 3060's advantage is cheap generation on models that fit in 12 GB, and the unlock does not change that. Neither card's VRAM changes, and VRAM still decides which models you can run at all.

Verdict matrix

Recommended pick for the long-context RAG reader: the RTX 4090. Its stock prefill is about 6× the 3060's on the scoreboard, its 24 GB holds a 32B model at Q4 with room for context, and if the unlock matures it gains on exactly the phase that dominates a 32K-token request. A 3060 owner chasing the same workload gets more from a smaller model, a better quant or KV-cache quantization than from the hack.

Hardware to complete the build

• MSI Gaming GeForce RTX 3060 12GB: the entry ticket to 12 GB of VRAM; generation runs at its bandwidth ceiling, so it suits chat-length prompts.
• ZOTAC Gaming GeForce RTX 3060 Twin Edge OC 12GB: the same GA106 chip in a compact two-fan card for small cases.
• MSI Gaming GeForce RTX 4090 SUPRIM Liquid X 24GB: the long-context pick; liquid cooling helps if an unlocked card draws more under sustained prefill.
• ASUS ROG Strix GeForce RTX 3090 OC 24GB: 24 GB at 936 GB/s for 30B-class models when a 4090 is out of budget.
• GIGABYTE GeForce RTX 4070 Ti Gaming OC 12GB: a newer Ada 12 GB option with far more prefill compute than a 3060, still capped at 12 GB.

Live price comparison

Compare current prices and specs side by side: RTX 3060 12GB vs RTX 4090.

Related guides

• RTX 4090 vs RTX 5090 for local LLM inference
• RTX 3060 12GB for local LLMs: the Reddit consensus
• Best local LLM for 12GB of VRAM
• llama.cpp vs vLLM for single-user chat

Citations and sources

All accessed 2026-10-27.

• Tom's Hardware: Nvidia RTX 20, 30 and 40-series GPUs hacked to remove a half-speed AI throttle
• Hardware Busters: RTX 4090 tensor unlock doubles FP16 throughput on Linux
• JustAResearcher/gpu-unlock-ada (GitHub)
• Zagitta/gpu-unlock (GitHub)
• NVIDIA Ada GPU Architecture whitepaper
• NVIDIA Ampere GA102 GPU Architecture whitepaper
• TechPowerUp: GeForce RTX 3060 12 GB specs
• llama.cpp CUDA performance scoreboard (GitHub discussion #15013)
• llama.cpp build documentation
• Hardware Corner: RTX 3060 12GB LLM benchmarks
• Hardware Corner: RTX 4090 LLM benchmarks
• vLLM documentation
• PyTorch CUDA semantics notes
• ExLlamaV2 (GitHub)

This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.