Skip to main content
NVIDIA B200
NVIDIA · GPU · Blackwell

NVIDIA B200 — Benchmarks & Specs

192 GB VRAM1000W TDP2024

Short answer: the NVIDIA B200 generates 30000 tokens/sec on gpt-oss-120b (NVFP4) under tgi in a single cited run, per NVIDIA Developer Blog; there are too few comparable runs on file for a median. All 27 models in the Will-It-Run matrix fit in its 192 GB of VRAM at their listed quantization with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so this is not the card's ceiling.

  • Single cited run 30000 tok/s gpt-oss-120b NVFP4, per NVIDIA Developer Blog
  • Fits in 192 GB all 27 matrix models → at ~8K context; the largest in the sizing matrix is DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run), and the Will-It-Run matrix has no larger model

Who should buy this instead: the NVIDIA B200's 192 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 192 GB, start at the 128 GB unified pick.

Bottom line: how fast is the NVIDIA B200?

In a batched serving run it reaches 30000.0 tokens/sec aggregate throughput (many concurrent requests, not one user's generation speed) running gpt-oss-120b at NVFP4 under tgi, per NVIDIA Developer Blog. In MLPerf Inference v5.1 — Llama 2 70B Offline it scores 102,725 tok/s, per Lambda AI / MLCommons MLPerf Inference v5.1. Its 192 GB of VRAM is the binding constraint for local inference: by capacity alone it should fit 70B-parameter open-weight models at Q4 without offloading (estimated from VRAM capacity, not a measured run).

Every measured figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the model-size range is an estimate from VRAM capacity. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

Read the full comparison → NVIDIA H200 vs B200 for LLM Inference: What Changes in 2026

Check what fits in 192 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the NVIDIA B200's 192 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 163.6 GB of the card's 206.2 GB (192 GiB) free. Check a model in the Will-It-Run matrix →

What fits in the NVIDIA B200's 192 GB, and what doesn't

7 of these 7 model files load whole into 192 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face in decimal GB, and the card's 192 GB (binary GiB) is 206.2 GB in the same units. Speeds are this card's single-user generation speed from cited community runs at the file listed; a row with no cited run says so rather than estimating one.

ModelQuantFits in 192 GB?Cited speed on this card
Llama 3.1 8B 4.9 GB fileQ4_K_MFits~201.2 GB left for contextNo public measurement on file
Qwen3 14B 9 GB fileQ4_K_MFits~197.2 GB left for contextNo public measurement on file
gpt-oss-20b 21B-A3.6B · 12.1 GB fileMXFP4Fits~194 GB left for contextNo public measurement on file
Qwen3 30B-A3B 18.6 GB fileQ4_K_MFits~187.6 GB left for contextNo public measurement on file
Qwen3 32B 19.8 GB fileQ4_K_MFits~186.4 GB left for contextNo public measurement on file
Gemma 3 27B 28.7 GB fileQ8_0Fits~177.4 GB left for contextNo public measurement on file
Llama 3.3 70B 42.5 GB fileQ4_K_MFits~163.6 GB left for contextNo public measurement on file

The NVIDIA B200 is a datacenter accelerator from the Blackwell family released in 2024 from NVIDIA. Key on-paper specs include 192 GB of HBM3e memory, 8000 GB/s of memory bandwidth, 1000W TDP. Data on this page draws on 3 synthetic benchmark results, 17 community AI inference reports (top 12 shown), compiled from NVIDIA TensorRT-LLM Performance Overview, Nebius / MLCommons MLPerf Inference v6.0, NVIDIA Developer Blog, Lambda AI / MLCommons MLPerf Inference v5.1, SemiAnalysis InferenceX, Surya Dantuluri; each table row links to its source. Read this page when shopping the NVIDIA B200, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Batched serving throughput

Aggregate output tokens per second on the NVIDIA B200 summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table.
Model Quantization Relative Tokens/sec (aggregate) VRAM used Source
llama3.1:8b — custom optimized serving (Modal) batched 99500.0 tok/s aggregate — Surya Dantuluri (sdan.net) 2025-11-04
gpt-oss-120b MXFP4 trt-llm batched 60000.0 tok/s aggregate — NVIDIA Developer Blog (SemiAnalysis InferenceMAX v1) 2025-10-09
gpt-oss:20b FP4 TensorRT-LLM batched 59084.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-04-16
gpt-oss:20b NVFP4 TensorRT-LLM (trtllm-bench, PyTorch backend) batched 53812.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-11-03
gpt-oss-120b NVFP4 tgi batched 30000.0 tok/s aggregate — NVIDIA Developer Blog / SemiAnalysis InferenceMAX 2025-10-13
gpt-oss:120b FP4 TensorRT-LLM batched 29061.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-04-16
qwen3:30b-a3b NVFP4 TensorRT-LLM (trtllm-bench, PyTorch backend) batched 26971.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-11-03
qwen3:30b-a3b FP4 TensorRT-LLM batched 26431.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-04-16
gpt-oss:120b NVFP4 TensorRT-LLM (trtllm-bench, PyTorch backend) batched 25943.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-11-03
llama2:70b FP4 trt-llm batched 11264.0 tok/s aggregate — NVIDIA Developer Blog 2024-08-28
llama3.3:70b NVFP4 trt-llm batched 10000.0 tok/s aggregate — NVIDIA Developer Blog (SemiAnalysis InferenceMAX v1) 2025-10-09
llama3.3:70b FP8 InferenceX (SemiAnalysis) serving sweep batched 6519.0 tok/s aggregate — SemiAnalysis InferenceX 2026-10-07

Synthetic Benchmarks

Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.

Synthetic benchmark scores for the NVIDIA B200 — higher is better. Each row links to the public database the number was taken from.
Benchmark Relative Score Source
MLPerf Inference v5.1 — Llama 2 70B Offline 102,725 tok/s Lambda AI / MLCommons MLPerf Inference v5.1 2025-09-09
MLPerf Inference v6.0 — gpt-oss 120B Offline 85,921 tok/s Nebius / MLCommons MLPerf Inference v6.0 2026-03-30
MLPerf Inference v6.0 — DeepSeek R1 Offline 58,582 tok/s Nebius / MLCommons MLPerf Inference v6.0 2026-03-30

Full Specifications

TDP1000 W
Segmentdata-center
VRAM192 GB
Memory typeHBM3e
CUDA cores16896
Process5 nm
Form factorSXM
L2 cache50 MB
ArchitectureBlackwell
InterconnectNVLink 5.0
Memory bandwidth8000 GB/s
Nvlink bandwidth tbps1.8

NVIDIA B200 — Frequently Asked Questions

What is the NVIDIA B200 best used for?
NVIDIA B200 is a 192 GB VRAM Blackwell-family data-center GPU. Server AI inference and training, with 192 GB of memory for large models, are the headline use cases; it is built for servers rather than gaming desktops; the benchmark tables on this page place it within its generation.
When was the NVIDIA B200 released?
NVIDIA B200 was released in 2024, a Blackwell-generation part.
Where do the NVIDIA B200 benchmark numbers come from?
Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.
Can the NVIDIA B200 run local LLMs?
Yes — AI inference results for the NVIDIA B200 are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 192 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.
Where can I buy the NVIDIA B200?
SpecPicks has no verified listing of the NVIDIA B200 on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the NVIDIA B200's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Editorial guides covering the NVIDIA B200

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →