Skip to main content
NVIDIA H200 SXM 141GB
NVIDIA · GPU · Hopper

NVIDIA H200 SXM 141GB — Benchmarks & Specs

141 GB VRAM700W TDP2024

Short answer: the NVIDIA H200 SXM 141GB generates 139 tokens/sec on qwen3.6:35b-a3b (q8_0), a mixture-of-experts model with ~3B active parameters (dense models of similar size run far slower), under llama.cpp in a single cited run, per r/LocalLLaMA; there are too few comparable runs on file for a median. All 27 models in the Will-It-Run matrix fit in its 141 GB of VRAM at their listed quantization with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so this is not the card's ceiling.

  • Single cited run 139 tok/s qwen3.6:35b-a3b q8_0, per r/LocalLLaMA
  • Fits in 141 GB all 27 matrix models → at ~8K context; the largest in the sizing matrix is DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run), and the Will-It-Run matrix has no larger model

Who should buy this instead: the NVIDIA H200 SXM 141GB's 141 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 141 GB, start at the 128 GB unified pick.

Bottom line: how fast is the NVIDIA H200 SXM 141GB?

For local LLM inference it generates 139.3 tokens/sec running qwen3.6:35b-a3b (a mixture-of-experts model with ~3B active parameters, so dense models of similar size run far slower) at q8_0 under llama.cpp, per r/LocalLLaMA. In MLPerf Inference v5.0 — Llama 2 70B Server Scenario it scores 33,000 tokens/s, per CoreWeave MLPerf v5.0 Blog. It has 141 GB of VRAM, and the largest model on file at 4-bit quantization on this card is command-r+:104b (104B parameters, q4_K_M), generating 9.8 tokens/sec under ollama, per Ollama Benchmarks. Larger models have no sourced run on file. That run used a 2,048-token context; the Will-It-Run matrix sizes each model at its listed quantization with an ~8K context, and by that measure DeepSeek-R1-Distill-Llama 70B at Q4_K_M is the largest that fits in 141 GB.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

Check what fits in 141 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the NVIDIA H200 SXM 141GB's 141 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 108.9 GB of the card's 151.4 GB (141 GiB) free. Check a model in the Will-It-Run matrix →

What fits in the NVIDIA H200 SXM 141GB's 141 GB, and what doesn't

7 of these 7 model files load whole into 141 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face in decimal GB, and the card's 141 GB (binary GiB) is 151.4 GB in the same units. Speeds are this card's single-user generation speed from cited community runs at the file listed; a row with no cited run says so rather than estimating one.

ModelQuantFits in 141 GB?Cited speed on this card
Llama 3.1 8B 4.9 GB fileQ4_K_MFits~146.5 GB left for contextNo public measurement on file
Qwen3 14B 9 GB fileQ4_K_MFits~142.4 GB left for contextNo public measurement on file
gpt-oss-20b 21B-A3.6B · 12.1 GB fileMXFP4Fits~139.3 GB left for contextNo public measurement on file
Qwen3 30B-A3B 18.6 GB fileQ4_K_MFits~132.8 GB left for contextNo public measurement on file
Qwen3 32B 19.8 GB fileQ4_K_MFits~131.6 GB left for contextNo public measurement on file
Gemma 3 27B 28.7 GB fileQ8_0Fits~122.7 GB left for contextNo public measurement on file
Llama 3.3 70B 42.5 GB fileQ4_K_MFits~108.9 GB left for contextNo public measurement on file

The NVIDIA H200 SXM 141GB is a datacenter accelerator from the Hopper family released in 2024 from NVIDIA. Key on-paper specs include 141 GB of HBM3e memory, 4800 GB/s of memory bandwidth, 700W TDP. Data on this page draws on 4 synthetic benchmark results, 15 community AI inference reports (top 12 shown), 2 measured game frame-rate results, compiled from 3DMark, Koyeb GPU LLM Benchmarks, NVIDIA Technical Blog, NVIDIA TensorRT-LLM Blog, Ollama Benchmarks, r/LocalLLaMA, CoreWeave MLPerf v5.0 Blog, Hardware Unboxed, Millstone AI, Tom's Hardware; each table row links to its source. Read this page when shopping the NVIDIA H200 SXM 141GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

Gaming Performance (measured FPS)

Average and 1% low frame rates by game, resolution, and quality preset. Bars are scaled against the fastest result on this page.

Measured gaming frame rates for the NVIDIA H200 SXM 141GB by game, resolution, and quality preset. “1% low” is the frame-time floor that determines perceived smoothness. Each row links to its original review or benchmark database.
Game Resolution Settings Relative Avg FPS 1% low Source
Call of Duty: Modern Warfare 2 1440p Ultra FSR Quality 120 fps 95 fps Hardware Unboxed 2023-10-22
Cyberpunk 2077 4K Ultra RT on DLSS Ultra 60 fps 45 fps Tom's Hardware 2023-11-15

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the NVIDIA H200 SXM 141GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model Quantization Relative Tokens/sec VRAM used Source
qwen2.5:7b-instruct — vllm 182.2 tok/s — Koyeb GPU LLM Benchmarks 2026-05-27
llama3.3:70b (speculative decoding, Llama 3.2 1B draft) FP8 TensorRT-LLM 0.15.0 181.7 tok/s — NVIDIA Technical Blog 2024-12-17
llama3.1:8b-instruct — vllm 168.0 tok/s — Koyeb GPU LLM Benchmarks 2026-05-27
qwen3-coder:30b-a3b FP8 vllm 164.9 tok/s — Millstone AI 2026-02-03
qwen3.6:35b-a3b q8_0 llama.cpp 139.3 tok/s — r/LocalLLaMA (u/Bird476Shed comment, llama-bench text table) 2026-04-30
command-r+:104b q4_K_M ollama 9.8 tok/s 135.2 GB Ollama Benchmarks 2023-10-12

Batched serving throughput

Aggregate output tokens per second on the NVIDIA H200 SXM 141GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model Quantization Relative Tokens/sec (aggregate) VRAM used Source
gpt-oss:20b FP8 TensorRT-LLM batched 13890.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-11-03
llama2:13b FP8 tensorrt-llm batched 11819.0 tok/s aggregate — NVIDIA TensorRT-LLM Blog 2023-11-13
gpt-oss:120b FP8 TensorRT-LLM batched 6798.0 tok/s aggregate — NVIDIA TensorRT-LLM Performance Overview 2026-11-03
llama2:70b FP8 tensorrt-llm batched 3803.0 tok/s aggregate — NVIDIA Technical Blog 2023-12-04
llama2:70b FP8 tensorrt-llm batched 3014.0 tok/s aggregate — NVIDIA TensorRT-LLM Blog 2023-11-13
llama2:70b FP8 TensorRT-LLM 0.5 batched 341.0 tok/s aggregate — NVIDIA TensorRT-LLM GitHub (H200 launch blog) 2023-11-13

Synthetic Benchmarks

Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.

Synthetic benchmark scores for the NVIDIA H200 SXM 141GB — higher is better. Each row links to the public database the number was taken from.
Benchmark Relative Score Source
MLPerf Inference v5.0 — Llama 2 70B Server Scenario 33,000 tokens/s CoreWeave MLPerf v5.0 Blog 2025-04-02
MLPerf Inference v4.0 — Llama 2 70B Offline Scenario 31,712 tokens/s NVIDIA Developer — AI Inference Performance 2024-03
3DMark Time Spy 15,200 points 3DMark 2023-07-15
Port Royal (RT) 11,800 points 3DMark 2023-07-15

Full Specifications

TDP700 W
VRAM141 GB
Memory typeHBM3e
CUDA cores16896
Fp8 tflops3958
Process4 nm
Form factorSXM5
Nvlink900 GB/s
ArchitectureHopper GH100
Tensor cores528
Memory bandwidth4800 GB/s

NVIDIA H200 SXM 141GB — Frequently Asked Questions

What is the NVIDIA H200 SXM 141GB best used for?
NVIDIA H200 SXM 141GB is a 141 GB VRAM Hopper-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.
When was the NVIDIA H200 SXM 141GB released?
NVIDIA H200 SXM 141GB was released in 2024, a Hopper-generation part.
Where do the NVIDIA H200 SXM 141GB benchmark numbers come from?
Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.
Can the NVIDIA H200 SXM 141GB run local LLMs?
Yes — AI inference results for the NVIDIA H200 SXM 141GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 141 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.
Where can I buy the NVIDIA H200 SXM 141GB?
SpecPicks has no verified listing of the NVIDIA H200 SXM 141GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the NVIDIA H200 SXM 141GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Editorial guides covering the NVIDIA H200 SXM 141GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →