Short answer: As of October 2026, LLM builders should care only if they need 70B–120B models in memory or serve many users at once, and even then the $9,999 QuietBox 2 has to beat a single 96GB card. Hardware Corner measured the RTX PRO 6000 Blackwell at 32.0 tok/s on Llama 3.3 70B and 210.1 tok/s on gpt-oss-120b for one user (Hardware Corner, March 2026), on a card it lists at an $8,800 average market value. The QuietBox 2 sells for $9,999 complete, with 128GB of GDDR6 across four Blackhole chips (Tenstorrent spec sheet). Its measured edge is concurrency: 35.9 tok/s per user with 32 simultaneous Gemma 4 31B requests (Tenstorrent, 25 September 2026). For one user on a 27–31B model a single RTX 5090 is still faster: 68.5 tok/s on Gemma 4 31B in a llama.cpp run against 46.5 tok/s on the QuietBox 2 (Tenstorrent). The RTX 5090 card alone was listed new on Amazon at $7,443.99 on October 11, 2026, up from $6,995.95 on September 28 and against a $1,999 MSRP. The first single-user result in the QuietBox 2's favor is a small mixture-of-experts model: a tt-metal change merged October 16, 2026 measured gpt-oss-20b at 352 tok/s per user at batch 1 on the QuietBox 2 (tt-metal PR #60161), against 298.2 tok/s for gpt-oss 20B on an RTX 5090 at 4k context (Hardware Corner).
Tenstorrent TT-QuietBox 2 (Blackhole) vs RTX 5090: Should LLM Builders Care?
Tenstorrent's $9,999 four-chip Blackhole workstation is shipping. A single RTX 5090 is faster for one user; the QuietBox 2 wins on memory capacity and concurrency.
As of Sept 2026 the $9,999 TT-QuietBox 2 runs Gemma 4 31B at 46.5 tok/s per user vs 68.5 on an RTX 5090, but holds 128GB and serves 32 users.
Check another model on the RTX 5090: Will Qwen2.5 32B run on the RTX 5090? → Runs, 45.1 tok/s from 1 cited run. Each page swaps the card or the model and lists every source.
Hardware at a Glance
Median generation throughput at 7–9B models (Llama 3.1 8B, Qwen 3 8B), Q4 quantization, from community-reported runs SpecPicks tracks. Each row pools runs from different sources, runtimes and models in that class, so the rows are not a matched head-to-head; where the article compares cards on the same rig, its own figures are the like-for-like result. Street price is the second-lowest listing priced within the last 24 hours at 0.5–2× launch MSRP, so no single listing sets it, and a street figure well above MSRP says so in its cell; where too few listings pass that check the cell shows a dash and the launch MSRP, which is not comparable with the street figures. Rows marked for comparison are not the cards this article is about — they are included so the throughput column has something to be read against.
| GPU | VRAM | Llama-3-8B class, Q4 | Street price | Sources |
|---|---|---|---|---|
| NVIDIA GeForce RTX 5090 | 32 GB | 224.8 tok/s4 runs · 3 sources | —too few current listings for a street figure · launch MSRP was $1,999 | SpecPicks median of 4 runs; sources: llama.cpp GitHub, DatabaseMart, Hardware Corner |
| NVIDIA GeForce RTX 3090 | 24 GB | 93.9 tok/s6 runs · 5 sources | $2,000street, all listings · 1.3× the $1,499 launch MSRP | SpecPicks median of 6 runs; sources: Hardware Corner, GitHub XiongjieDai/GPU-Benchmarks…, kunalganglani.com LLM Benchmarks, LocalScore (Mozilla Builders) +1 more |
| Radeon RX 7900 XTXfor comparison | 24 GB | 138.6 tok/s9 runs · 3 sources | $1,995street, all listings · 2.0× the $999 launch MSRP | SpecPicks median of 9 runs; sources: GitHub / 1337hero, llama.cpp GitHub issue #20934, Markaicode / LocalScore |
Which models fit on an RTX 5090?
RTX 5090 carries 32 GB of VRAM. The weights column is the range of real Q4_K_M files on Hugging Face for the models in each size class (about 0.6 GB per billion parameters), and the runtime plus a usable context window wants about 2 GB on top; every tokens-per-second figure is a median over community-reported Q4 runs SpecPicks tracks for this card, with the run count and the source beside it.
| Model size | Weights at Q4 | Fits in 32 GB? | Measured | Left for context | Source |
|---|---|---|---|---|---|
| 3B (Llama 3.2 3B, Qwen 3 4B)Runs on almost anything with a discrete GPU, and usably on modern integrated graphics. | about 2 GB | Fits: weights and a usable context window | Nothing on file → | ~29 GBfor runtime and KV cache | — |
| 7-9B (Llama 3.1 8B, Qwen 3 8B)The mainstream local model. An 8 GB card fits it; a 12 GB card fits it with real context. | 4.4–5.8 GB | Fits: weights and a usable context window | 224.8 tok/s4 runs · 3 sources | ~26 GBfor runtime and KV cache | Hardware Corner |
| 12-14B (Qwen 3 14B, Phi-4)Where 8 GB stops being enough. 12 GB is the smallest card that holds it with a usable context window. | 7.3–9.1 GB | Fits: weights and a usable context window | 89.9 tok/s3 runs · 2 sources | ~22 GBfor runtime and KV cache | Hardware Corner |
| 20-27B (Gemma 3 27B, Mistral Small)A 27B Q4_K_M file (about 16.7 GB) is more than a 16 GB card holds, so 16 GB means a Q3 quant or partial CPU offload; a 24B loads with a short context. 20 GB holds the class with a usable context window, 24 GB with a long one. | 14.3–16.7 GB | Fits: weights and a usable context window | Nothing on file → | ~15 GBfor runtime and KV cache | — |
| 30-35B (Qwen 3 32B, QwQ 32B)The step change. A 24 GB card holds this entirely in VRAM; below that it is CPU offload. | about 19.9 GB | Fits: weights and a usable context window | 55.6 tok/s8 runs · 2 sources | ~12 GBfor runtime and KV cache | Hardware Corner |
| 70B+ (Llama 3.3 70B, Qwen 2.5 72B)One 48 GB card or two 24 GB cards. A 32 GB card runs it only with layers in system RAM. | about 42.5 GB | No: spills to system RAM — system-RAM bandwidth sets the speed | 17.8 tok/s3 runs · 3 sources | none | LocalLLaMA |
Every RTX 5090 benchmark run, with its source → GPU picks for local LLM — the same table across every card we track How we source these numbers
What changed since October 2026's first update (figures to October 11): Tenstorrent merged a gpt-oss-20b decode rewrite on October 16, 2026 that cut batch-1 decode from 18.35 ms to 2.47 ms per token on the QuietBox 2 and reports 2,400 tok/s total at batch 32 (tt-metal PR #60161), and tt-inference-server v0.25.0 (October 15, 2026) moved Gemma 4 31B and Qwen3.8-27B on the QuietBox 2 to "COMPLETE" (tt-inference-server releases).
The full cross-card comparison: Best GPUs for Running Local LLMs in 2026 has the VRAM-tier table, the largest model each card fits at Q4, and measured tok/s with a source link on every row.
What's changed as of September 2026
This article first ran in April 2026, before the QuietBox 2 shipped. Much of what it said then was wrong:
- It shipped, and it costs less than stated here. Tenstorrent announced the QuietBox 2 on March 11, 2026 for "Q2 2026" at "starting at $9,999". The product page now lists it at $9,999 with "In stock: Ships in 2 weeks". The earlier version of this article said $14,999.
- The hardware is not what this article described. Per the official specifications, it uses two p300c cards (two Blackhole chips each), not four p150c cards, and an AMD Ryzen 7 9700X host, not a Threadripper. It has 256GB of DDR5-5600 and is liquid-cooled at 38 dBA, not passively cooled. Memory bandwidth is 512 GB/s per chip (2,048 GB/s total), not 1 TB/s per card, and maximum system draw is 1,300W, not ~3,000W.
- The "476.5 tokens per second" headline is a 32-user total. The launch press release says Llama 3.1 70B "runs at 476.5 tokens per second". Tenstorrent's model matrix lists that figure for batch 32 on the earlier four-card QuietBox (Blackhole). That is 14.9 tok/s per user.
- Tenstorrent now publishes measured QuietBox 2 numbers for current models. Gemma 4 31B, Qwen3.8-27B and Llama 3.1 8B each have a dedicated QuietBox 2 implementation with measured per-user speed (table below).
- The RTX 5090 is no longer a $2,000 card. A price tracker recorded a low of $4,199.99 on July 30 and $6,995.95 on September 28, 2026. Tech Insider reported $4,329 on Amazon and $4,299–$5,300 at Micro Center in mid-September.
- Fabricated material has been removed. The previous version cited a llama.cpp "Tenstorrent backend PR #9876". #9876 is an unrelated feature request for the Aria model. It also named "TT-Metal 0.55" and "vLLM-tt 0.4" releases and had a projected tok/s table, a quantization matrix, a context-latency table and perf-per-dollar math. None of those had a source, and all are replaced below with cited measurements or removed.
What changed — October 2026
A second pass added the comparison the September version was missing:
- New hardware: the RTX PRO 6000 Blackwell. A single 96GB GDDR7 card with 1,790 GB/s of bandwidth and a 600W TDP (Hardware Corner), the one-card way to hold 70B–120B models that the QuietBox 2 is sold on.
- New table of single-user decode speeds on 70B–123B models: 32.0 tok/s on Llama 3.3 70B, 210.1 tok/s on gpt-oss-120b and 19.1 tok/s on Mistral 123B at 4k context (Hardware Corner), cross-checked against StorageReview's 31.74 and 163.15 tok/s (StorageReview).
- Same-source RTX 5090 baseline. Hardware Corner measured Gemma 4 31B at 61.1 tok/s on the 5090 and 61.6 tok/s on the PRO 6000 (Hardware Corner 5090, Hardware Corner PRO 6000).
- Cost per GB. Hardware Corner prices the PRO 6000 at $91.67 per GB of VRAM (Hardware Corner), against about $78 per GB for the QuietBox 2.
- Opening answer rewritten to weigh the QuietBox 2 against the 96GB single-card option.
- The RTX 5090 got more expensive again. The price tracker shows it new on Amazon at $7,350.00 on October 9 and $7,443.99 on October 11, 2026, up from $6,995.95 on September 28. That is more than 3.7 times its $1,999 MSRP, which narrows the gap to the $9,999 QuietBox 2.
- Removed an unsupported date. Hardware Corner's $8,800 average market value for the RTX PRO 6000 Blackwell has no date on the page, so the opening no longer calls it an October 2026 price.
What changed — mid-October 2026
- A QuietBox 2 result that beats the RTX 5090 for one user. Tenstorrent's PR #60161, merged October 16, 2026, reworks gpt-oss-20b on the QuietBox 2 (four Blackhole chips, tensor parallel 4). Its description reports batch-1 decode at 128 input tokens dropping from 18.35 ms to 2.47 ms per token, "404.98 tok/s/user" in the demo. A later commit on the same branch re-timed plain batch-1 decode at 2.84 ms/token (352 tok/s per user). Hardware Corner measured gpt-oss 20B (Q4_K) on an RTX 5090 at 298.2 tok/s at 4k context (Hardware Corner). The context lengths and runtimes differ, so this is a direction, not a head-to-head.
- Batch-32 gpt-oss-20b. A commit in the same PR reports "13.33 ms/step (75.0 tok/s/user, 2400 tok/s)" at batch 32 (PR #60161).
- Status promotions. tt-inference-server v0.25.0 (October 15, 2026) lists Qwen3.8-27B and gemma-4-31B-it on P300X2 (QuietBox 2) as "COMPLETE", up from "Experimental" when this article last updated (tt-inference-server releases). gpt-oss-120b on the QuietBox 2 is still Experimental.
- New stable tt-metal. v0.80.0 shipped October 14, 2026 (tt-metal releases), with release notes that include "Add Qwen3.8 27B QB2" and "Add Gemma4 31B QB2". The gpt-oss-20b change above merged after it, so it is in main and nightly builds, not yet a stable release.
Current-model decode speed: TT-QuietBox 2 vs RTX 5090 (September 2026)
Batch-1 decode is bounded by memory bandwidth divided by the bytes of weights read per token. The QuietBox 2 splits each model across four chips (tensor parallel 4), so its ceiling uses the full 2,048 GB/s. The RTX 5090 has 32GB of GDDR7 at 1,792 GB/s. For the QuietBox 2, bytes per token are estimated from the Hugging Face BF16 checkpoint size, scaled to Tenstorrent's block-float formats (block-FP4 is roughly a quarter of BF16, block-FP8 roughly half). For the RTX 5090 they are the summed Hugging Face GGUF file for the exact quant.
| Model | Hardware | Weight format | Weights read per token | Bandwidth ceiling | Measured decode, 1 user | Source |
|---|---|---|---|---|---|---|
| Gemma 4 31B | TT-QuietBox 2 | Block-FP4 projections, block-FP8 LM head | ~17.6 GB (from 62.55 GB BF16) | ~116 tok/s | 46.5 tok/s | Tenstorrent QB2 demo, measured Sept 25, 2026 |
| Gemma 4 31B | RTX 5090 | Q4_K_M GGUF | 18.7 GB (17.40 GiB file) | ~96 tok/s | 68.5 tok/s | Atomic Chat, llama.cpp b10988, Sept 18, 2026 |
| Qwen3.8-27B | TT-QuietBox 2 | Block-FP4 on every weight group (precision config) | ~15.6 GB (from 55.56 GB BF16) | ~131 tok/s | 39.4 tok/s | Tenstorrent QB2 demo |
| Qwen3.8-27B | RTX 5090 | UD-Q4_K_XL GGUF | 17.56 GB | ~102 tok/s | 73.2 tok/s (plain decode, no MTP) | RTX 5090 report, Unsloth GGUF thread, Aug 14, 2026 |
| Llama 3.1 8B | TT-QuietBox 2 | Block-FP8 / block-FP4 mix | ≤8.5 GB (from 16.06 GB BF16) | ≥240 tok/s | 131.1 tok/s | Tenstorrent QB2 CI run |
| gpt-oss-20b | TT-QuietBox 2 | Tenstorrent formats, TP=4 | — | — | 352 tok/s (2.84 ms/token, 128-token prompt) | tt-metal PR #60161, merged Oct 16, 2026 |
| gpt-oss 20B | RTX 5090 | Q4_K GGUF | — | — | 298.2 tok/s (4k context) | Hardware Corner, March 2026 |
| Llama 3.1 70B | TT-QuietBox (Blackhole), 4× p150 | Block-FP8 or smaller | ≤75 GB (from 141.1 GB BF16) | ≥27 tok/s | 14.9 tok/s per user at batch 32 (476.5 tok/s total) | Tenstorrent model matrix |
Every measured figure sits under its ceiling. The RTX 5090 reaches about 70% of its ceiling on both 27–31B models. The QuietBox 2 reaches 30–40%, because each generated token needs cross-chip reductions over the Ethernet fabric. The Register saw the same pattern in November 2025 on the earlier Blackhole QuietBox, reporting about 41% of peak theoretical performance on a 70B Llama model across four cards. Speculative decoding adds to the 5090's lead. The same Qwen3.8-27B thread reports 1.2–1.5× more throughput with llama.cpp's MTP drafting enabled.
The 70B and 120B tier is where the comparison flips. Llama 3.3 70B at Q4_K_M is a 42.52 GB file, and gpt-oss-120b in MXFP4 is 63.39 GB. Neither fits in the 5090's 32GB without spilling to system RAM. Both fit in the QuietBox 2's 128GB. The press release says "GPT-OSS 120B runs entirely on-device".
Concurrency and prefill: where the QuietBox 2 earns its price
Tenstorrent's Gemma 4 31B serving measurements (25 September 2026, one QuietBox 2, vLLM plugin) cover both one user and 32:
| Server slots / concurrent requests | Input tokens | TTFT | Decode per user | Completed requests/s |
|---|---|---|---|---|
| 1 / 1 | 128 | 68 ms | 46.5 tok/s | 0.357 |
| 1 / 1 | 1,024 | 147 ms | 44.3 tok/s | 0.332 |
| 1 / 1 | 32,768 | 5.8 s | 42.4 tok/s | — |
| 32 / 32 | 128 | 1.9 s | 35.9 tok/s | 5.847 |
| 32 / 32 | 1,024 | 3.1 s | 21.0 tok/s | 3.402 |
| 32 / 32 | 2,049 | 7.7 s | 15.4 tok/s | 1.863 |
Two things stand out. First, 32 users at 35.9 tok/s each is roughly 1,150 tok/s of total output on a 31B model, from one desktop box. The 5090 figures above come from single-slot llama.cpp runs, and this synthesis found no comparable published 32-user measurement for the card. Second, the old claim that the QuietBox has "half the 5090's prefill" does not match current data. A 32,768-token prompt reaches first token in 5.8 s on the QuietBox 2, about 5,600 tok/s of prefill. The Atomic Chat run measured the 5090 at 3,698 tok/s on a 4k prompt for the same model. The engines and prompt lengths differ, so treat this as a sign that prefill is no longer the QuietBox's weak spot, not a head-to-head result.
The single-card alternative: RTX PRO 6000 Blackwell (96GB)
The QuietBox 2's case rests on holding 70B–120B models in accelerator memory, which a 32GB RTX 5090 cannot do. Hardware Corner's RTX 5090 page answers "no" to whether the card can run 70B models (Hardware Corner). NVIDIA's RTX PRO 6000 Blackwell can. It is one card with 96GB of GDDR7, 1,790 GB/s of memory bandwidth and a 600W TDP, at an $8,800 average market price on Hardware Corner's listing in October 2026.
| Model (quant) | Hardware | Context or prompt | Decode, 1 user | Source |
|---|---|---|---|---|
| Gemma 4 31B (Q4_K) | RTX PRO 6000 Blackwell | 4k context | 61.6 tok/s | Hardware Corner, March 2026 |
| Gemma 4 31B (Q4_K) | RTX 5090 | 4k context | 61.1 tok/s | Hardware Corner, March 2026 |
| Gemma 4 31B (block-FP4) | TT-QuietBox 2 | 128 input tokens | 46.5 tok/s | Tenstorrent, 25 September 2026 |
| Llama 3.3 70B (Q4_K) | RTX PRO 6000 Blackwell | 4k context | 32.0 tok/s | Hardware Corner |
| Llama 3.3 70B (Q4_K) | RTX PRO 6000 Blackwell | 128k context | 16.6 tok/s | Hardware Corner |
| Llama 3.3 70B | RTX PRO 6000 Blackwell | 49-token prompt, LM Studio | 31.74 tok/s | StorageReview, October 2025 |
| Llama 3.1 70B | TT-QuietBox (Blackhole), 4× p150 | Batch 32 | 14.9 tok/s per user | Tenstorrent model matrix |
| gpt-oss-120b (MXFP4) | RTX PRO 6000 Blackwell | 4k context | 210.1 tok/s | Hardware Corner |
| gpt-oss-120b (MXFP4) | RTX PRO 6000 Blackwell | 128k context | 99.8 tok/s | Hardware Corner |
| gpt-oss-120b | RTX PRO 6000 Blackwell | 81-token prompt, LM Studio | 163.15 tok/s, 0.193 s to first token | StorageReview |
| gpt-oss-120b | TT-QuietBox 2 | Batch 1 only | No published tok/s; rated "Experimental" | tt-inference-server |
| Mistral 123B (Q4_K) | RTX PRO 6000 Blackwell | 4k context | 19.1 tok/s | Hardware Corner |
Two things follow. First, on the same Gemma 4 31B test, the PRO 6000 decodes about as fast as a 5090 (61.6 against 61.1 tok/s), so its extra 64GB costs no single-user speed. Second, for one user on 70B-class models, the only Blackhole figure Tenstorrent publishes is the 14.9 tok/s per user at batch 32, measured on the earlier four-card box. This synthesis found no published single-user QuietBox 2 result on a 70B model. The PRO 6000's 32.0 tok/s is measured at batch 1, so the two figures are not a head-to-head.
The costs differ in shape. Hardware Corner works the PRO 6000 out at $91.67 per GB of VRAM (Hardware Corner), against about $78 per GB for the QuietBox 2, and the card still needs a host PC that the QuietBox 2 includes. Power is close. StorageReview measured a PRO 6000 workstation at 918.5 W average and 1,036.3 W peak system draw under a Stable Diffusion XL load (StorageReview), and the QuietBox 2 is rated at up to 1,300W (Tenstorrent spec sheet).
Pick the RTX PRO 6000 over the QuietBox 2 if one user or a small team runs 70B–120B models and wants CUDA, GGUF and day-one model support. Pick the QuietBox 2 if the job is serving dozens of concurrent requests, where its 32-user measurements have no published PRO 6000 equivalent here.
What is the TT-QuietBox 2, and how does it differ from the first Blackhole QuietBox?
The QuietBox 2 is a liquid-cooled desktop workstation built around four Tenstorrent Blackhole chips on two p300c cards. Tenstorrent's spec sheet lists 480 Tensix cores at 1.35 GHz and 128GB of GDDR6 (32GB per chip). Each chip has 512 GB/s of memory bandwidth, and chips are linked by 800 Gbps Ethernet. The host is a Ryzen 7 9700X with 256GB of DDR5-5600 and a 4TB NVMe drive on an ASRock B850M-C board, powered by a 1,600W PSU. The p300c card is not sold separately. The launch release rates the system at 2,654 TFLOPS of BlockFP8 compute and says it plugs into a standard 120V outlet. The spec sheet adds that it can draw up to 1,300W, so it is best kept off a shared circuit.
It replaces two earlier QuietBoxes, both now listed as sold out on Tenstorrent's product page. The first Blackhole QuietBox cost $11,999 and, per The Register's review, used four separate Blackhole P150c cards. The Wormhole QuietBox cost $15,000 and used four Wormhole n300 cards with 96GB in total. For single-card builds, Tenstorrent sells the Blackhole p150a/p150b for $1,399: 120 Tensix cores, 32GB of GDDR6 at 512 GB/s, 664 TFLOPS BlockFP8, and 300W board power.
Spec comparison
| Spec | TT-QuietBox 2 | NVIDIA RTX 5090 |
|---|---|---|
| Accelerator | 4× Blackhole chips on 2× p300c cards (spec sheet) | One GB202 GPU (TechPowerUp) |
| Accelerator memory | 128GB GDDR6 | 32GB GDDR7 |
| Memory bandwidth | 2,048 GB/s total (512 GB/s per chip) | 1,792 GB/s |
| Host included | Ryzen 7 9700X, 256GB DDR5-5600, 4TB NVMe | No (card only) |
| Power | Up to 1,300W system; 1,600W PSU | 575W total graphics power; 1,000W system PSU required (NVIDIA) |
| Noise | 38 dBA under max load | Varies by board partner |
| Price (September 2026) | $9,999 complete, in stock (Tenstorrent) | $1,999 MSRP; $4,299–$6,995.95 street for the card (videocardprices.com, Tech Insider) |
| Software | TT-Metalium, TT-Forge, vLLM plugin, tt-inference-server | CUDA; every mainstream runtime |
The 2026 price spike changed the memory math. At $9,999 the QuietBox 2 costs about $78 per GB of accelerator memory. A 5090 at the $4,329 Amazon price is about $135 per GB, or about $219 per GB at the $6,995.95 listing, and that is before the rest of the PC.
Software stack reality check — September 2026
What works today:
- TT-Metalium is under active release. The latest stable build is v0.80.0 (October 14, 2026), following v0.78.0 (September 5, 2026), with nightly dev builds in between.
- vLLM support now comes through the vLLM TT plugin. The tt-metal README says the earlier vLLM fork is deprecated.
- tt-inference-server is Tenstorrent's supported serving path. Its QuietBox 2 model list rates Gemma 4 31B, Gemma 4 12B and Qwen3.8-27B as "Complete" as of v0.25.0 (October 15, 2026). Llama 3.3 70B, Qwen3-32B and Llama 3.1 8B are "Functional". Qwen3.6-27B and gpt-oss-120b are "Experimental", and gpt-oss-120b is limited to batch size 1. Image and video models (FLUX.1-dev, FLUX.1-schnell, Wan2.2) and Whisper are rated "Complete"; Mochi was dropped from the QuietBox 2 list on October 12, 2026 ("remove incorrect support for mochi-1-preview on QB2").
- Out of the box, the QuietBox 2 guide says the system "ships with Qwen3-32B pre-loaded".
What is still rough:
- No upstream llama.cpp support. The community discussion on a Tenstorrent backend (llama.cpp #10858) has a maintained fork but no upstream merge. The fork's author notes the KV cache has to stay on the CPU. Ollama, LM Studio and GGUF files therefore don't run on it natively. You serve through vLLM or tt-inference-server with Tenstorrent's own weight formats.
- New models arrive later than on CUDA, and as experimental. Qwen3.8-27B and Gemma 4 31B each needed a dedicated QuietBox 2 implementation, and gpt-oss-120b's multi-batch path on 8× p150 was still an open pull request in September 2026. On a CUDA card, community GGUF quants run in llama.cpp without a device-specific port. Unsloth's Qwen3.8-27B GGUF thread had RTX 5090 results by August 14, 2026.
- Polish. In November 2025, The Register concluded that the earlier Blackhole QuietBox's software "simply isn't polished enough for most local AI enthusiasts". The QuietBox 2 demos and guide are a clear step forward, and as of October 2026 the two headline 27–31B models are rated "Complete", but gpt-oss-120b and Qwen3.6-27B still carry "Experimental" labels.
Verdict matrix
Get the TT-QuietBox 2 if:
- You need 70B-class or gpt-oss-120b models fully in accelerator memory, and $9,999 for 128GB beats building a multi-GPU rig at 2026 GPU prices.
- You serve several users or agents at once. 32 concurrent Gemma 4 31B requests at 35.9 tok/s each is the measured sweet spot.
- You are comfortable with vLLM and Tenstorrent's supported model list, and can wait for "Experimental" models to mature.
- You want an open-source stack from compiler to kernel, or a non-NVIDIA option, for strategic reasons.
Stick with the RTX 5090 if:
- Your models are dense and 31B or smaller. It decodes 68.5–73.2 tok/s on current 27–31B models versus 39.4–46.5 on the QuietBox 2, for one user.
- You want GGUF support in llama.cpp, Ollama or LM Studio, plus speculative decoding.
- You also fine-tune, or run ComfyUI or other CUDA-only tools.
- You can find the card near its $4,299–$4,329 street floor, not the $7,000 listings.
Wait if:
- Your target model is still "Experimental" on the QuietBox 2 list, or you only need one user on a model that fits in 32GB.
Bottom line
As of September 2026 the TT-QuietBox 2 is a real, shipping $9,999 product, not a roadmap slide. It is the cheapest way here to get 128GB of fast accelerator memory in one desktop box. It is still not a faster RTX 5090. For one user on a current 27–31B model the 5090 is roughly 1.5–1.9× quicker (68.5 vs 46.5 tok/s on Gemma 4 31B, 73.2 vs 39.4 on Qwen3.8-27B), and CUDA's day-one model support is unmatched. Buy the QuietBox 2 for 70B–120B models and for serving many users at once. Buy the 5090 for one fast chat session on models that fit in 32GB.
Related guides
- Best GPU for AI workstations in 2026
- Best 24GB GPU for local LLM inference in 2026
- RTX 5090 vs Mac Studio M4 Max for AI
- AMD Ryzen AI Max+ 395 (Strix Halo) for local LLMs
- Dual Radeon AI PRO R9700 workstation build
Citations and sources
- Tenstorrent, TT-QuietBox product page ($9,999 QuietBox 2 in stock; earlier Blackhole and Wormhole QuietBoxes sold out): https://tenstorrent.com/en/hardware/tt-quietbox
- Tenstorrent, TT-QuietBox 2 (Blackhole) specifications (2× p300c, Ryzen 7 9700X, 256GB DDR5, 2,048 GB/s, 1,300W max, 38 dBA): https://docs.tenstorrent.com/systems/quietbox/quietbox-bh-2/specifications.html
- Tenstorrent, TT-QuietBox 2 launch press release via Yahoo Finance (March 11, 2026; $9,999; 2,654 TFLOPS BlockFP8; 476.5 tok/s Llama 3.1 70B claim): https://finance.yahoo.com/news/tenstorrent-unveils-tt-quietbox-tm-160000561.html
- Tenstorrent, Blackhole card lineup (p150a/p150b $1,399, 512 GB/s, 664 TFLOPS BlockFP8, 300W): https://tenstorrent.com/en/hardware/blackhole
- Tenstorrent, QuietBox 2 guide (ships with Qwen3-32B pre-loaded): https://docs.tenstorrent.com/tt-quietbox2-guide/
- tt-metal, Gemma 4 31B on QuietBox 2, measured serving results (September 25, 2026): https://github.com/tenstorrent/tt-metal/blob/main/models/demos/gemma4_31b_qb2/README.md
- tt-metal, Qwen3.8-27B on QuietBox 2 (39.4 tok/s per user at batch 1): https://github.com/tenstorrent/tt-metal/blob/main/models/demos/qwen38_27b_qb2/README.md
- tt-metal, Qwen3.8-27B QuietBox 2 precision configuration (block-FP4 weights): https://github.com/tenstorrent/tt-metal/blob/main/models/demos/qwen38_27b_qb2/config/precision.json
- tt-metal, Llama 3.1 8B on QuietBox 2 (131.08 tok/s per user in CI): https://github.com/tenstorrent/tt-metal/blob/main/models/demos/llama31_8b_qb2/README.md
- tt-metal, model performance matrix (Llama 3.1 70B, QuietBox Blackhole, batch 32: 14.9 tok/s per user, 476.5 tok/s total): https://github.com/tenstorrent/tt-metal/blob/main/models/README.md
- tt-metal, v0.78.0 release (September 5, 2026): https://github.com/tenstorrent/tt-metal/releases/tag/v0.78.0
- tt-metal README (vLLM fork deprecated in favor of the vLLM TT plugin): https://github.com/tenstorrent/tt-metal
- tt-metal pull request #55589, GPT-OSS multi-batch prefill and decode on Blackhole (open, September 2026): https://github.com/tenstorrent/tt-metal/pull/55589
- tt-metal pull request #60161, gpt-oss-20b on QuietBox 2 batch-1 decode 18.35 -> 2.47 ms/token, 352 tok/s per user plain decode, 2,400 tok/s at batch 32 (merged October 16, 2026): https://github.com/tenstorrent/tt-metal/pull/60161
- tt-metal releases, v0.80.0 stable (October 14, 2026; adds Qwen3.8 27B and Gemma4 31B QB2): https://github.com/tenstorrent/tt-metal/releases
- tt-inference-server releases, v0.25.0 (October 15, 2026; Gemma 4 31B and Qwen3.8-27B on P300X2 rated COMPLETE): https://github.com/tenstorrent/tt-inference-server/releases
- tt-inference-server, commit history for models by hardware (October 12, 2026 removal of mochi-1-preview on QB2): https://github.com/tenstorrent/tt-inference-server/commits/main/docs/model_support/models_by_hardware.md
- tt-inference-server, models by hardware (QuietBox 2 support status): https://github.com/tenstorrent/tt-inference-server/blob/main/docs/model_support/models_by_hardware.md
- Tenstorrent, vLLM TT plugin: https://github.com/tenstorrent/vllm-tt-plugin
- The Register, "Blackhole QuietBox, Tenstorrent's AI workstation reviewed" (November 27, 2025): https://www.theregister.com/on-prem/2025/11/27/blackhole-quietbox-tenstorrents-ai-workstation-reviewed/2113269
- llama.cpp discussion #10858, Tenstorrent backend (not merged upstream): https://github.com/ggml-org/llama.cpp/discussions/10858
- llama.cpp issue #9876 (an Aria model feature request, not a Tenstorrent backend PR): https://github.com/ggml-org/llama.cpp/issues/9876
- Atomic Chat, best local LLMs for RTX 5090, measured on llama.cpp b10988 (September 18, 2026): https://atomic.chat/blog/guides/best-local-llms-for-rtx-5090
- Hugging Face, unsloth/Qwen3.8-27B-GGUF RTX 5090 performance thread (August 2026): https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/discussions/14
- Hugging Face, unsloth/Qwen3.8-27B-GGUF file listing (UD-Q4_K_XL 17.56 GB): https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
- Hugging Face, google/gemma-4-31B-it (BF16 checkpoint size): https://huggingface.co/google/gemma-4-31B-it
- Hugging Face, Qwen/Qwen3.8-27B (BF16 checkpoint size): https://huggingface.co/Qwen/Qwen3.8-27B
- Hugging Face, meta-llama/Llama-3.1-8B-Instruct (BF16 checkpoint size): https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct
- Hugging Face, meta-llama/Llama-3.1-70B-Instruct (BF16 checkpoint size): https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct
- Hugging Face, unsloth/Llama-3.3-70B-Instruct-GGUF (Q4_K_M 42.52 GB): https://huggingface.co/unsloth/Llama-3.3-70B-Instruct-GGUF
- Hugging Face, ggml-org/gpt-oss-120b-GGUF (MXFP4 63.39 GB): https://huggingface.co/ggml-org/gpt-oss-120b-GGUF
- TechPowerUp, GeForce RTX 5090 specifications (32GB GDDR7, 512-bit, 1,792 GB/s): https://www.techpowerup.com/gpu-specs/geforce-rtx-5090.c4216
- NVIDIA, GeForce RTX 5090 product page (575W total graphics power, 1,000W system power): https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
- videocardprices.com, RTX 5090 price tracker (September 2026): https://videocardprices.com/card/nvidia-rtx-5090/
- Tech Insider, RTX 5090 price by channel (September 16, 2026): https://tech-insider.org/rtx-5090-price-4329-rtx-60-delay-2028-2026/
- Hardware Corner, RTX PRO 6000 Blackwell LLM benchmarks (llama.cpp, updated March 2026; 96GB, 1,790 GB/s, 600W TDP, $8,800 average market price, $91.67 per GB): https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-pro-6000-blackwell/
- Hardware Corner, RTX 5090 LLM benchmarks (llama.cpp, updated March 2026; Gemma 4 31B 61.1 tok/s; gpt-oss 20B 298.2 tok/s at 4k; no 70B in VRAM): https://www.hardware-corner.net/gpu-llm-benchmarks/rtx-5090/
- StorageReview, NVIDIA RTX PRO 6000 workstation GPU review (October 7, 2025; LM Studio results, system power draw): https://www.storagereview.com/review/nvidia-rtx-pro-6000-workstation-gpu-review-blackwell-architecture-and-96-gb-for-pro-workflows
This piece is editorial synthesis based on publicly available information. No independent first-party benchmarking is reported.
Products mentioned
Products mentioned in this article
Amazon & eBay listings, plus full specs and alternatives on each product page.
As an Amazon Associate, SpecPicks earns from qualifying purchases; we also earn on qualifying eBay purchases via the eBay Partner Network. Prices shown were last tracked at crawl time and may vary — check the listing for the current price.
Watch a review
Gigabyte Aorus Master RTX 5090 Review — Hardware Unboxed on YouTube
Frequently asked questions
Is the TT-QuietBox 2 faster than an RTX 5090 for local LLMs?
How much does the TT-QuietBox 2 cost, and is it shipping?
What does the 476.5 tokens per second figure for Llama 3.1 70B mean?
What hardware is inside the TT-QuietBox 2?
How mature is Tenstorrent's software stack in September 2026?
Related reading
- Mistral Medium 3.5 128B on Local Hardware: MLX 4-bit at ~70GB Explained
- RTX 5090 vs RTX Spark for Local LLMs: 32 GB GDDR7 vs 128 GB Unified (2026)
- RTX 5090 AI Build Guide: CPU, RAM, PSU & Cooling for Local Inference
- RTX 5090 Prebuilt vs a $700 RTX 3060 Local-LLM Box: What Extra VRAM Actually Buys
- MiMo-V2.5-Pro Local Hardware Requirements: VRAM, Tok/s, and Quantization on Consumer GPUs
- Kimi-Dev-72B Local Coding Benchmarks: VRAM Required, Tok/s, and How It Stacks Against DeepSeek V4 and…
