Updated 2026-09-23 23 models × 19 GPUs 239 cited runs CSV (CC BY 4.0)

Will It Run? Local LLM × GPU Matrix

Name: SpecPicks Will-It-Run matrix: local LLM × GPU fit and speed
Creator: SpecPicks
License: https://creativecommons.org/licenses/by/4.0/

Qwen3 32B at Q4_K_M is a 19.8 GB file (per its Hugging Face listing), so a 12 GB RTX 3060 cannot hold it and a 24 GB card can. This matrix runs that same arithmetic for 23 open-weight models on 19 GPUs, then attaches the median of 239 published, source-linked speed measurements where they exist (129 of 437 pairs so far).

By SpecPicks Editorial — AI Rigs Team

Check one pair

The matrix

Each cell links to its own page with the answer, the file-size arithmetic, every cited run and a buy link for the card. Runs = fits with 1.5 GB+ to spare; Tight fit = loads with a short context; Offload = part of the model spills to system RAM. A number is the median published generation speed, with the run count in brackets.

Model	40608 GB	3060 Ti8 GB	3060 12GB12 GB	B58012 GB	4070 SUPER12 GB	4060 Ti 16GB16 GB	5060 Ti 16GB16 GB	A770 16GB16 GB	RX 9070 XT16 GB	4070 Ti SUPER16 GB	408016 GB	5070 Ti16 GB	508016 GB	309024 GB	RX 7900 XTX24 GB	409024 GB	509032 GB	A6000 48GB48 GB	PRO 6000 Blackwell 96GB96 GB
Llama 3.2 1B0.8 GB Q4_K_M	Runs	Runs	Runs184 tok/s (1)	Runs	Runs183 tok/s (1)	Runs209 tok/s (1)	Runs210 tok/s (3)	Runs	Runs	Runs161 tok/s (1)	Runs	Runs185 tok/s (1)	Runs103 tok/s (1)	Runs267 tok/s (1)	Runs106 tok/s (1)	Runs	Runs	Runs	Runs244 tok/s (1)
Llama 3.2 3B2 GB Q4_K_M	Runs	Runs	Runs126 tok/s (2)	Runs	Runs	Runs	Runs	Runs64.2 tok/s (1)	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs
Mistral 7B4.4 GB Q4_K_M	Runs50.9 tok/s (1)	Runs71.2 tok/s (1)	Runs	Runs14.4 tok/s (3)	Runs	Runs	Runs90.2 tok/s (1)	Runs41.5 tok/s (2)	Runs	Runs	Runs	Runs112 tok/s (1)	Runs	Runs	Runs	Runs	Runs	Runs	Runs
Qwen2.5 7B4.7 GB Q4_K_M	Runs42.2 tok/s (1)	Runs58.1 tok/s (1)	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs
Llama 3.1 8B4.9 GB Q4_K_M	Runs41.7 tok/s (1)	Runs57.3 tok/s (1)	Runs53.6 tok/s (4)	Runs40 tok/s (5)	Runs53.4 tok/s (3)	Runs48.2 tok/s (3)	Runs59.2 tok/s (7)	Runs38.7 tok/s (1)	Runs	Runs53.9 tok/s (1)	Runs101 tok/s (4)	Runs95.3 tok/s (2)	Runs88.5 tok/s (2)	Runs92 tok/s (3)	Runs48.3 tok/s (2)	Runs126 tok/s (6)	Runs150 tok/s (1)	Runs	Runs138 tok/s (1)
DeepSeek-R1-Distill-Llama 8B4.9 GB Q4_K_M	Runs41.3 tok/s (1)	Runs	Runs	Runs13.7 tok/s (1)	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs
Qwen3 8B5 GB Q4_K_M	Runs	Runs	Runs42 tok/s (8)	Runs24.2 tok/s (2)	Runs65.8 tok/s (2)	Runs34.3 tok/s (4)	Runs51.2 tok/s (2)	Runs21 tok/s (3)	Runs90 tok/s (3)	Runs63.4 tok/s (4)	Runs77.9 tok/s (5)	Runs121 tok/s (1)	Runs94.1 tok/s (1)	Runs101 tok/s (2)	Runs	Runs	Runs186 tok/s (1)	Runs	Runs
Gemma 2 9B5.8 GB Q4_K_M	Runs18 tok/s (1)	Runs23.8 tok/s (1)	Runs	Runs38 tok/s (1)	Runs	Runs	Runs	Runs23.6 tok/s (2)	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs
Gemma 3 12B7.3 GB Q4_K_M	Tight fit	Tight fit	Runs	Runs31.4 tok/s (2)	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs71 tok/s (1)	Runs	Runs	Runs	Runs	Runs	Runs
Qwen2.5 14B9 GB Q4_K_M	Offload	Offload	Runs26.4 tok/s (1)	Runs35 tok/s (2)	Runs24.9 tok/s (1)	Runs25.6 tok/s (1)	Runs32.8 tok/s (4)	Runs21.4 tok/s (1)	Runs	Runs36.8 tok/s (1)	Runs	Runs55.1 tok/s (2)	Runs55.3 tok/s (2)	Runs55.8 tok/s (3)	Runs46.8 tok/s (2)	Runs63.9 tok/s (1)	Runs89.9 tok/s (1)	Runs50.3 tok/s (1)	Runs81.9 tok/s (1)
DeepSeek-R1-Distill-Qwen 14B9 GB Q4_K_M	Offload	Offload	Runs29.4 tok/s (1)	Runs	Runs	Runs	Runs	Runs13.4 tok/s (1)	Runs	Runs	Runs	Runs	Runs70 tok/s (1)	Runs37.5 tok/s (1)	Runs	Runs59 tok/s (3)	Runs89.1 tok/s (1)	Runs48.4 tok/s (1)	Runs
Qwen3 14B9 GB Q4_K_M	Offload	Offload	Runs28.1 tok/s (2)	Runs	Runs41.4 tok/s (2)	Runs22.4 tok/s (5)	Runs33 tok/s (2)	Runs23.2 tok/s (3)	Runs50.1 tok/s (2)	Runs47.2 tok/s (3)	Runs62 tok/s (2)	Runs66.2 tok/s (2)	Runs64 tok/s (1)	Runs61.1 tok/s (2)	Runs	Runs82.8 tok/s (1)	Runs124 tok/s (1)	Runs40.7 tok/s (1)	Runs
Phi-4 14B9.1 GB Q4_K_M	Offload	Offload	Runs	Runs	Runs	Runs	Runs	Runs22.3 tok/s (2)	Runs	Runs	Runs	Runs	Runs	Runs	Runs	Runs68.8 tok/s (2)	Runs	Runs52.6 tok/s (1)	Runs
gpt-oss-20b12.1 GB MXFP4	Offload	Offload	Offload	Offload	Offload	Runs	Runs	Runs	Runs91.9 tok/s (1)	Runs98.7 tok/s (5)	Runs138 tok/s (2)	Runs	Runs134 tok/s (2)	Runs	Runs	Runs	Runs	Runs	Runs
Gemma 3 27B16.6 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs	Runs	Runs	Runs47.3 tok/s (1)	Runs	Runs61.5 tok/s (1)
Gemma 2 27B16.7 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs	Runs	Runs38 tok/s (2)	Runs	Runs31.6 tok/s (1)	Runs
Qwen3 30B-A3B18.6 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs	Runs	Runs196 tok/s (1)	Runs234 tok/s (1)	Runs	Runs
Qwen3 32B19.8 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs32.7 tok/s (2)	Runs27 tok/s (1)	Runs38 tok/s (3)	Runs59.7 tok/s (2)	Runs27.5 tok/s (3)	Runs56 tok/s (1)
Qwen2.5 32B19.9 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs23 tok/s (2)	Runs28 tok/s (1)	Runs34.4 tok/s (1)	Runs45.1 tok/s (1)	Runs26.1 tok/s (1)	Runs
DeepSeek-R1-Distill-Qwen 32B19.9 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs	Runs26 tok/s (1)	Runs34.2 tok/s (1)	Runs49.8 tok/s (2)	Runs26.2 tok/s (1)	Runs64.3 tok/s (1)
Llama 3.1 70B42.5 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs14.6 tok/s (1)	Runs30.5 tok/s (1)
Llama 3.3 70B42.5 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs13.6 tok/s (1)	Runs32 tok/s (3)
DeepSeek-R1-Distill-Llama 70B42.5 GB Q4_K_M	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Offload	Runs13.9 tok/s (2)	Runs30.2 tok/s (2)

Download the data

The whole matrix — file sizes, VRAM, bandwidth, verdict, bandwidth ceiling, median speed, run count and every source URL — is one CSV: specpicks.com/will-it-run.csv. It is published under Creative Commons Attribution 4.0; credit "SpecPicks Will-It-Run matrix" with a link to this page.

How the checks work

Fit is the GGUF file size from Hugging Face against the card's VRAM, with 1.5 GB reserved for the runtime and an ~8K context. Speeds come from the SpecPicks benchmark database, and a published run is dropped when the pair does not fit (it would be measuring system RAM), when it is not Q4-class, when its source labels it an estimate, when it is faster than the card's memory bandwidth divided by the bytes read per token, or when one source reports the identical figure on four or more different cards.

Frequently asked questions

How does the matrix decide whether a model will run?

It compares the model's real download size (the Q4_K_M GGUF file on Hugging Face, or MXFP4 for gpt-oss) against the card's VRAM. With 1.5 GB or more to spare it runs; with 0.5–1.5 GB it is a tight fit that needs a short context; below that, part of the model is offloaded to system RAM and speed collapses to what RAM and PCIe allow.

Where do the tokens-per-second figures come from?

From published runs in the SpecPicks benchmark database, each linked to its source (llama.cpp discussions, LocalScore, Hardware Corner, vendor blogs and similar). A figure is quoted only when the pair fits in VRAM, the run is Q4-class, it is not labelled an estimate, it sits under the memory-bandwidth ceiling for that card, and it is not one number pasted across many cards. The cell shows the median and how many runs it rests on.

Why is there no speed for some pairs that fit?

Because no published run for that exact pair survived the checks. The fit verdict is still arithmetic on real file sizes, so it stands on its own, but a speed is only printed when someone measured it. Those pages are kept out of search indexes until a measurement exists.

Can I use the data?

Yes. The full matrix is a CSV at /will-it-run.csv under Creative Commons Attribution 4.0: use it for anything, including commercially, with a link back to specpicks.com/will-it-run.

Where to go next

Which GPU for which LLM — VRAM floors and a card to buy per model tier.
The GPU picker — rank cards by VRAM floor and budget.
AI rigs hub — complete local-AI builds.

As an Amazon Associate, SpecPicks earns from qualifying purchases.

This page is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.