Skip to main content
NVIDIA H100 PCIe 80GB
NVIDIA · GPU · Hopper

NVIDIA H100 PCIe 80GB — Benchmarks & Specs

80 GB VRAM350W TDP2022

Short answer: the NVIDIA H100 PCIe 80GB generates a median 146 tokens/sec on 7-9B dense models at 4-bit (Q4) under llama.cpp, across 3 cited runs from llama.cpp GitHub Discussions and GPU-Benchmarks-on-LLM-Inference. All 27 models in the Will-It-Run matrix fit in its 80 GB of VRAM at their listed quantization with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so this is not the card's ceiling. An eBay listing for it is $26,801.32, checked (listings can end at any time).

  • Median speed 146 tok/s 3 cited runs, 7-9B at Q4, llama.cpp
  • Fits in 80 GB all 27 matrix models → at ~8K context; the largest in the sizing matrix is DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run), and the Will-It-Run matrix has no larger model

Who should buy this instead: the NVIDIA H100 PCIe 80GB's 80 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 80 GB, start at the 128 GB unified pick.

NVIDIA H100 PCIe 80GB: verdict by use case

  • For everyday local models (7-9B dense): a median 146 tok/s at 4-bit (Q4) under llama.cpp, across 3 cited runs (llama.cpp GitHub Discussions, GPU-Benchmarks-on-LLM-Inference).
  • For large models in 80 GB: all 27 models in the Will-It-Run matrix fit with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so 80 GB has headroom beyond it that is not sized here.

Each verdict quotes a figure from a linked source or a row in the tables below. Where no cited figure exists for a use case, there is no verdict line for it.

Bottom line: how fast is the NVIDIA H100 PCIe 80GB?

SpecPicks median: 146 tokens/sec across 3 cited single-user runs of 7-9B dense models at 4-bit (Q4) under llama.cpp (range 144–281 tok/s), from llama.cpp GitHub Discussions, GPU-Benchmarks-on-LLM-Inference (2 runs). For local LLM inference it generates 28.3 tokens/sec running qwen2:72b at q4_K_M under ollama, per DatabaseMart. In 3DMark Time Spy it scores 2,681 points, per TechSpot. Its 80 GB of VRAM is the binding constraint for local inference: the largest model on file at 4-bit quantization on this card is qwen2.5:110b (110B parameters, q4_K_M), generating 20.2 tokens/sec under ollama, per DatabaseMart. That run's source does not report a context length; the Will-It-Run matrix sizes each model at its listed quantization with an ~8K context, and by that measure DeepSeek-R1-Distill-Llama 70B at Q4_K_M is the largest that fits in 80 GB.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the SpecPicks median is computed here from the runs it names, each a row in the AI table. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

eBay listing (New): $26,801.32, checked . eBay listings are third-party and can end at any time; as an eBay Partner Network member, SpecPicks may earn a commission.

Check what fits in 80 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the NVIDIA H100 PCIe 80GB's 80 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 43.4 GB of the card's 85.9 GB (80 GiB) free. Check a model in the Will-It-Run matrix →

What fits in the NVIDIA H100 PCIe 80GB's 80 GB, and what doesn't

7 of these 7 model files load whole into 80 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face in decimal GB, and the card's 80 GB (binary GiB) is 85.9 GB in the same units. Speeds are this card's single-user generation speed from cited community runs at the file listed; a row with no cited run says so rather than estimating one.

ModelQuantFits in 80 GB?Cited speed on this card
Llama 3.1 8B 4.9 GB fileQ4_K_MFits~81 GB left for contextNo public measurement on file
Qwen3 14B 9 GB fileQ4_K_MFits~76.9 GB left for contextNo public measurement on file
gpt-oss-20b 21B-A3.6B · 12.1 GB fileMXFP4Fits~73.8 GB left for contextNo public measurement on file
Qwen3 30B-A3B 18.6 GB fileQ4_K_MFits~67.3 GB left for contextNo public measurement on file
Qwen3 32B 19.8 GB fileQ4_K_MFits~66.1 GB left for contextNo public measurement on file
Gemma 3 27B 28.7 GB fileQ8_0Fits~57.2 GB left for contextNo public measurement on file
Llama 3.3 70B 42.5 GB fileQ4_K_MFits~43.4 GB left for context24.3 tok/s1 cited run, ollama, per DatabaseMart

The NVIDIA H100 PCIe 80GB is a datacenter accelerator from the Hopper family released in 2022 from NVIDIA. Key on-paper specs include 80 GB of HBM2e memory, 350W TDP. Data on this page draws on 1 synthetic benchmark result, 28 community AI inference reports (top 14 shown), 1 measured game frame-rate result, compiled from DatabaseMart, GPU-Benchmarks-on-LLM-Inference, llama.cpp GitHub Discussions, TechSpot, Markaicode, Spheron Blog; each table row links to its source. Read this page when shopping the NVIDIA H100 PCIe 80GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

Gaming Performance (measured FPS)

Average and 1% low frame rates by game, resolution, and quality preset. Bars are scaled against the fastest result on this page.

Measured gaming frame rates for the NVIDIA H100 PCIe 80GB by game, resolution, and quality preset. “1% low” is the frame-time floor that determines perceived smoothness. Each row links to its original review or benchmark database.
Game Resolution Settings Relative Avg FPS 1% low Source
Red Dead Redemption 2 1440p High 8 fps — TechSpot 2023-06-21

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the NVIDIA H100 PCIe 80GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model Quantization Relative Tokens/sec VRAM used Source
llama2:7b q4_0 llama.cpp 280.7 tok/s — llama.cpp GitHub Discussions 2024-01
llama3:8b Q4_K_M llama.cpp 145.6 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-12
llama3:8b q4_K_M llama.cpp 144.5 tok/s — GPU-Benchmarks-on-LLM-Inference (GitHub, XiongjieDai) 2024-05
llama3.3:70b FP8 vllm 120.0 tok/s 76.0 GB Spheron Blog 2026-01
phi3:mini q4_K_M ollama 88.1 tok/s 9.0 GB Markaicode 2026-07-07
deepseek-r1:14b Q4_K_M ollama 75.0 tok/s — DatabaseMart 2025-02
llama3:8b FP16 llama.cpp 68.0 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-12
llama3:8b FP16 llama.cpp 67.8 tok/s — GPU-Benchmarks-on-LLM-Inference (GitHub, XiongjieDai) 2024-05
qwen2:72b Q4_K_M ollama 28.3 tok/s — DatabaseMart 2025-02
qwen2.5:110b q4_K_M ollama 20.2 tok/s — DatabaseMart 2026-07-08

Batched serving throughput

Aggregate output tokens per second on the NVIDIA H100 PCIe 80GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model Quantization Relative Tokens/sec (aggregate) VRAM used Source
deepseek-r1-distill-qwen:7b FP16 vllm batched 6270.0 tok/s aggregate 72.0 GB DatabaseMart 2025-03
deepseek-r1-distill-llama:8b FP16 vllm batched 5561.0 tok/s aggregate 72.0 GB DatabaseMart 2025-03
deepseek-r1-distill-qwen:14b FP16 vllm batched 3689.2 tok/s aggregate — DatabaseMart 2026-08-25
deepseek-r1-distill-qwen:14b FP16 vllm batched 3689.0 tok/s aggregate 72.0 GB DatabaseMart 2025-03

Synthetic Benchmarks

Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.

Synthetic benchmark scores for the NVIDIA H100 PCIe 80GB — higher is better. Each row links to the public database the number was taken from.
Benchmark Relative Score Source
3DMark Time Spy 2,681 points TechSpot 2023-06-21Same result reported by Appuals (citing Geekerwan)

Full Specifications

TDP350 W
VRAM80 GB
CUDA cores14592
Memory typeHBM2e

NVIDIA H100 PCIe 80GB — Frequently Asked Questions

What is the NVIDIA H100 PCIe 80GB best used for?
NVIDIA H100 PCIe 80GB is an 80 GB VRAM Hopper-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.
When was the NVIDIA H100 PCIe 80GB released?
NVIDIA H100 PCIe 80GB was released in 2022, a Hopper-generation part.
Where do the NVIDIA H100 PCIe 80GB benchmark numbers come from?
Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.
Can the NVIDIA H100 PCIe 80GB run local LLMs?
Yes — AI inference results for the NVIDIA H100 PCIe 80GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 80 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.
Where can I buy the NVIDIA H100 PCIe 80GB?
SpecPicks has no verified listing of the NVIDIA H100 PCIe 80GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the NVIDIA H100 PCIe 80GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Editorial guides covering the NVIDIA H100 PCIe 80GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →