NVIDIA B200 — Benchmarks & Specs
Short answer: the NVIDIA B200 generates 30000 tokens/sec on gpt-oss-120b (NVFP4) under tgi in a single cited run, per NVIDIA Developer Blog; there are too few comparable runs on file for a median. All 27 models in the Will-It-Run matrix fit in its 192 GB of VRAM at their listed quantization with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so this is not the card's ceiling.
- Single cited run 30000 tok/s gpt-oss-120b NVFP4, per NVIDIA Developer Blog
- Fits in 192 GB all 27 matrix models → at ~8K context; the largest in the sizing matrix is DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run), and the Will-It-Run matrix has no larger model
Who should buy this instead: the NVIDIA B200's 192 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 192 GB, start at the 128 GB unified pick.
Bottom line: how fast is the NVIDIA B200?
In a batched serving run it reaches 30000.0 tokens/sec aggregate throughput (many concurrent requests, not one user's generation speed) running gpt-oss-120b at NVFP4 under tgi, per NVIDIA Developer Blog. In MLPerf Inference v5.1 — Llama 2 70B Offline it scores 102,725 tok/s, per Lambda AI / MLCommons MLPerf Inference v5.1. Its 192 GB of VRAM is the binding constraint for local inference: by capacity alone it should fit 70B-parameter open-weight models at Q4 without offloading (estimated from VRAM capacity, not a measured run).
Every measured figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the model-size range is an estimate from VRAM capacity. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.
Read the full comparison → NVIDIA H200 vs B200 for LLM Inference: What Changes in 2026
Check what fits in 192 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the NVIDIA B200's 192 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 163.6 GB of the card's 206.2 GB (192 GiB) free. Check a model in the Will-It-Run matrix →
What fits in the NVIDIA B200's 192 GB, and what doesn't
7 of these 7 model files load whole into 192 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face in decimal GB, and the card's 192 GB (binary GiB) is 206.2 GB in the same units. Speeds are this card's single-user generation speed from cited community runs at the file listed; a row with no cited run says so rather than estimating one.
| Model | Quant | Fits in 192 GB? | Cited speed on this card |
|---|---|---|---|
| Llama 3.1 8B 4.9 GB file | Q4_K_M | Fits~201.2 GB left for context | No public measurement on file |
| Qwen3 14B 9 GB file | Q4_K_M | Fits~197.2 GB left for context | No public measurement on file |
| gpt-oss-20b 21B-A3.6B · 12.1 GB file | MXFP4 | Fits~194 GB left for context | No public measurement on file |
| Qwen3 30B-A3B 18.6 GB file | Q4_K_M | Fits~187.6 GB left for context | No public measurement on file |
| Qwen3 32B 19.8 GB file | Q4_K_M | Fits~186.4 GB left for context | No public measurement on file |
| Gemma 3 27B 28.7 GB file | Q8_0 | Fits~177.4 GB left for context | No public measurement on file |
| Llama 3.3 70B 42.5 GB file | Q4_K_M | Fits~163.6 GB left for context | No public measurement on file |
The NVIDIA B200 is a datacenter accelerator from the Blackwell family released in 2024 from NVIDIA. Key on-paper specs include 192 GB of HBM3e memory, 8000 GB/s of memory bandwidth, 1000W TDP. Data on this page draws on 3 synthetic benchmark results, 17 community AI inference reports (top 12 shown), compiled from NVIDIA TensorRT-LLM Performance Overview, Nebius / MLCommons MLPerf Inference v6.0, NVIDIA Developer Blog, Lambda AI / MLCommons MLPerf Inference v5.1, SemiAnalysis InferenceX, Surya Dantuluri; each table row links to its source. Read this page when shopping the NVIDIA B200, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).
AI Inference Performance
Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.
Batched serving throughput
| Model | Quantization | Relative | Tokens/sec (aggregate) | VRAM used | Source |
|---|---|---|---|---|---|
| llama3.1:8b | — custom optimized serving (Modal) batched | 99500.0 tok/s aggregate | — | Surya Dantuluri (sdan.net) 2025-11-04 | |
| gpt-oss-120b | MXFP4 trt-llm batched | 60000.0 tok/s aggregate | — | NVIDIA Developer Blog (SemiAnalysis InferenceMAX v1) 2025-10-09 | |
| gpt-oss:20b | FP4 TensorRT-LLM batched | 59084.0 tok/s aggregate | — | NVIDIA TensorRT-LLM Performance Overview 2026-04-16 | |
| gpt-oss:20b | NVFP4 TensorRT-LLM (trtllm-bench, PyTorch backend) batched | 53812.0 tok/s aggregate | — | NVIDIA TensorRT-LLM Performance Overview 2026-11-03 | |
| gpt-oss-120b | NVFP4 tgi batched | 30000.0 tok/s aggregate | — | NVIDIA Developer Blog / SemiAnalysis InferenceMAX 2025-10-13 | |
| gpt-oss:120b | FP4 TensorRT-LLM batched | 29061.0 tok/s aggregate | — | NVIDIA TensorRT-LLM Performance Overview 2026-04-16 | |
| qwen3:30b-a3b | NVFP4 TensorRT-LLM (trtllm-bench, PyTorch backend) batched | 26971.0 tok/s aggregate | — | NVIDIA TensorRT-LLM Performance Overview 2026-11-03 | |
| qwen3:30b-a3b | FP4 TensorRT-LLM batched | 26431.0 tok/s aggregate | — | NVIDIA TensorRT-LLM Performance Overview 2026-04-16 | |
| gpt-oss:120b | NVFP4 TensorRT-LLM (trtllm-bench, PyTorch backend) batched | 25943.0 tok/s aggregate | — | NVIDIA TensorRT-LLM Performance Overview 2026-11-03 | |
| llama2:70b | FP4 trt-llm batched | 11264.0 tok/s aggregate | — | NVIDIA Developer Blog 2024-08-28 | |
| llama3.3:70b | NVFP4 trt-llm batched | 10000.0 tok/s aggregate | — | NVIDIA Developer Blog (SemiAnalysis InferenceMAX v1) 2025-10-09 | |
| llama3.3:70b | FP8 InferenceX (SemiAnalysis) serving sweep batched | 6519.0 tok/s aggregate | — | SemiAnalysis InferenceX 2026-10-07 |
Synthetic Benchmarks
Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.
| Benchmark | Relative | Score | Source |
|---|---|---|---|
| MLPerf Inference v5.1 — Llama 2 70B Offline | 102,725 tok/s | Lambda AI / MLCommons MLPerf Inference v5.1 2025-09-09 | |
| MLPerf Inference v6.0 — gpt-oss 120B Offline | 85,921 tok/s | Nebius / MLCommons MLPerf Inference v6.0 2026-03-30 | |
| MLPerf Inference v6.0 — DeepSeek R1 Offline | 58,582 tok/s | Nebius / MLCommons MLPerf Inference v6.0 2026-03-30 |
Full Specifications
| TDP | 1000 W |
|---|---|
| Segment | data-center |
| VRAM | 192 GB |
| Memory type | HBM3e |
| CUDA cores | 16896 |
| Process | 5 nm |
| Form factor | SXM |
| L2 cache | 50 MB |
| Architecture | Blackwell |
| Interconnect | NVLink 5.0 |
| Memory bandwidth | 8000 GB/s |
| Nvlink bandwidth tbps | 1.8 |
NVIDIA B200 — Frequently Asked Questions
What is the NVIDIA B200 best used for?
When was the NVIDIA B200 released?
Where do the NVIDIA B200 benchmark numbers come from?
Can the NVIDIA B200 run local LLMs?
Where can I buy the NVIDIA B200?
Buying guides that rank the NVIDIA B200's class
This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.
Editorial guides covering the NVIDIA B200
In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.
- NVIDIA H200 vs B200 for LLM Inference: What Changes in 2026
- Tesla P40 vs RTX 3090: Used 24GB GPUs for Local LLMs in 2026
- Used RTX 3090 vs New GPU for Local LLMs: 2026 Reddit Consensus
- Best GPU for Local LLMs by VRAM: 2026 Reddit Consensus
- RTX 3060 12GB Local LLM: Reddit Consensus on What Runs (2026)
- gpt-oss on a 12GB GPU: 2026 Community Consensus (RTX 3060)
- RTX PRO 6000 vs RTX 5090 for Local 70B LLMs: 96GB vs 32GB
- Qwen3-Coder-Next 80B-A3B on a 12GB RTX 3060: Per-Quant VRAM, RAM and tok/s
- Browse all SpecPicks reviews →
More guides & deep dives from the SpecPicks archive
Browse all articles & guides →- RTX 4070 Super vs RX 7800 XT — Which to Buy in 2026
- Best Retro Handhelds in 2026: Pocket, Mid-Tier and FPGA Picks
- The Complete Voodoo5 5500 AGP Driver Guide (2026 Edition)
- Best 1440p Gaming GPUs in 2026
- How to Build a Windows 98 Retro PC in 2026
- Best Budget Gaming PC Build 2026 — ~$1,000 Parts List
- Emulation Hardware in 2026: FPGA, Software, and Cart-Reader Ecosystems
More reviews from the SpecPicks archive
Browse all reviews →- Wooden CPU Cooler: How to Build One and What to Expect
- Best Gaming SSD in 2026: 5 Picks for PS5, Steam Deck, and Custom Builds
- Best SATA SSD to Upgrade a PlayStation 4 Pro in 2026
- GameNative Unlocks Up to 100 FPS PC Gaming on Android
- Best GPU for Llama 3.1 70B (2026)
- Ex-Sega Engineer Builds a 'Super-Realistic' CRT Emulator
- Best CPU Cooler for the Ryzen 7 5800X: Air vs 120mm vs 240mm AIO
- Samsung Odyssey OLED G9 Drops $700 — and a Cheaper 4K Path
- Best Budget GPU for Local LLMs in 2026: RTX 3060 12GB vs the Alternatives
- Anthropic May Follow Microsoft to AMD: What It Means for Home AI Rigs
- Best budget SSD for a gaming PC in 2026: BX500 vs 970 EVO Plus vs SSD Plus
- Best CPU for a Budget AI + Gaming Rig: Ryzen 7 5700X vs 5800X vs 5600G
- Intel Arc AI Training in 2026: What Workloads Actually Work
- Retro PC storage in 2026: CompactFlash-to-IDE vs a SATA SSD for Win98/XP
- GameSir G7 SE vs 8BitDo Pro 2 for PC Gaming: Wired Precision vs Wireless Flexibility
- Retro Gear Cables Melting Into Cases: Causes and Fixes
- Raspberry Pi 4 as an SMB1 File Server for Windows 98 Retro Rigs
- Raspberry Pi 5 8GB Review: Specs, Price & Use Cases (2025)
- Ryzen 7 5800X vs i7-9700K: Better Home Workstation in 2026?
- LattePanda Sigma vs Raspberry Pi 4 8GB: x86 Windows SBC vs ARM Linux for Edge AI Workloads in 2026
- Linux vs Windows 11: AMD Ryzen 9 9950X3D Performance
- Anthropic's 2GW AMD Deal: Should You Still Run Local?
- Best Raspberry Pi Starter Kits for Beginners (2026)
- Best Apartment Sim Racing Wheel Setup: Quiet and Clamp-On (2026)
More buying guides from SpecPicks
Browse all buying guides →- Best CPUs for Gaming in 2026
- Best GPUs for Running Local LLMs in 2026
- Best DDR5 RAM for Gaming PCs in 2026
- Best NVMe SSDs for Gaming in 2026
- Best Retro Gaming Consoles & Handhelds for 2026
- Best Graphics Cards for Gaming in 2026
- Best Controllers for PC Gaming in 2026
- Best AM5 Motherboards for 2026
- Best External SSDs for Content Creators in 2026
- Best GPU for Running 27B-32B Local LLMs in 2026
- Best CPU Coolers for 2026
- Best 1440p 240Hz Gaming Monitors in 2026
- Best CPUs for Content Creators in 2026
- Best Mechanical Keyboards for Gaming in 2026
- Best Tools for Building and Repairing Retro PCs in 2026
- Best NVMe External Enclosures for 2026
- Best Gaming Monitors for 2026
- Best 4K Monitors for Content Creators in 2026
- Best Gaming Mice for 2026
- Best GPUs for 4K Gaming in 2026
- Best PC Cases for Building in 2026
Hardware benchmark data on SpecPicks
All benchmarks →- Raspberry Pi 4 8GB — benchmarks & specs
- AMD Ryzen 9 5900H — benchmarks & specs
- AMD Ryzen 5 7520C — benchmarks & specs
- AMD Ryzen 7 7840U — benchmarks & specs
- Radeon RX 6600 — benchmarks & specs
- NVIDIA RTX PRO 5000 Blackwell — benchmarks & specs
- Ryzen 7 5800X3D — benchmarks & specs
- AMD Ryzen 5 PRO 7645 — benchmarks & specs
- AMD Ryzen Threadripper PRO 7975WX — benchmarks & specs
- Radeon RX 6750 GRE 10GB — benchmarks & specs
- Radeon RX 9070 XT — benchmarks & specs
- AMD Ryzen 5 PRO 7640HS — benchmarks & specs
- Ryzen 5 5625U — benchmarks & specs
- GeForce 3 Ti 500 — benchmarks & specs
- AMD Ryzen 7 7700X — benchmarks & specs
- AMD Ryzen 3 PRO 7330U — benchmarks & specs
- Ryzen 5 7600 — benchmarks & specs
- RTX5000-Ada-4Q — benchmarks & specs
- Ryzen 3 7320U — benchmarks & specs
- Intel Arc B570 — benchmarks & specs
- Intel 865 Chipset Motherboard — benchmarks & specs
- Radeon RX 7600 XT — benchmarks & specs
- Quadro RTX 4000 — benchmarks & specs
- GRID RTX6000-1B — benchmarks & specs