NVIDIA H100 PCIe 80GB — Benchmarks & Specs
Short answer: the NVIDIA H100 PCIe 80GB generates a median 146 tokens/sec on 7-9B dense models at 4-bit (Q4) under llama.cpp, across 3 cited runs from llama.cpp GitHub Discussions and GPU-Benchmarks-on-LLM-Inference. All 27 models in the Will-It-Run matrix fit in its 80 GB of VRAM at their listed quantization with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so this is not the card's ceiling. An eBay listing for it is $26,801.32, checked (listings can end at any time).
- Median speed 146 tok/s 3 cited runs, 7-9B at Q4, llama.cpp
- Fits in 80 GB all 27 matrix models → at ~8K context; the largest in the sizing matrix is DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run), and the Will-It-Run matrix has no larger model
Who should buy this instead: the NVIDIA H100 PCIe 80GB's 80 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 80 GB, start at the 128 GB unified pick.
NVIDIA H100 PCIe 80GB: verdict by use case
- For everyday local models (7-9B dense): a median 146 tok/s at 4-bit (Q4) under llama.cpp, across 3 cited runs (llama.cpp GitHub Discussions, GPU-Benchmarks-on-LLM-Inference).
- For large models in 80 GB: all 27 models in the Will-It-Run matrix fit with an ~8K context, the largest in the sizing matrix being DeepSeek-R1-Distill-Llama 70B at Q4_K_M (estimated to fit, not a measured run); the matrix does not include larger models, so 80 GB has headroom beyond it that is not sized here.
Each verdict quotes a figure from a linked source or a row in the tables below. Where no cited figure exists for a use case, there is no verdict line for it.
Bottom line: how fast is the NVIDIA H100 PCIe 80GB?
SpecPicks median: 146 tokens/sec across 3 cited single-user runs of 7-9B dense models at 4-bit (Q4) under llama.cpp (range 144–281 tok/s), from llama.cpp GitHub Discussions, GPU-Benchmarks-on-LLM-Inference (2 runs). For local LLM inference it generates 28.3 tokens/sec running qwen2:72b at q4_K_M under ollama, per DatabaseMart. In 3DMark Time Spy it scores 2,681 points, per TechSpot. Its 80 GB of VRAM is the binding constraint for local inference: the largest model on file at 4-bit quantization on this card is qwen2.5:110b (110B parameters, q4_K_M), generating 20.2 tokens/sec under ollama, per DatabaseMart. That run's source does not report a context length; the Will-It-Run matrix sizes each model at its listed quantization with an ~8K context, and by that measure DeepSeek-R1-Distill-Llama 70B at Q4_K_M is the largest that fits in 80 GB.
Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the SpecPicks median is computed here from the runs it names, each a row in the AI table. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.
eBay listing (New): $26,801.32, checked . eBay listings are third-party and can end at any time; as an eBay Partner Network member, SpecPicks may earn a commission.
Check what fits in 80 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the NVIDIA H100 PCIe 80GB's 80 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 43.4 GB of the card's 85.9 GB (80 GiB) free. Check a model in the Will-It-Run matrix →
What fits in the NVIDIA H100 PCIe 80GB's 80 GB, and what doesn't
7 of these 7 model files load whole into 80 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face in decimal GB, and the card's 80 GB (binary GiB) is 85.9 GB in the same units. Speeds are this card's single-user generation speed from cited community runs at the file listed; a row with no cited run says so rather than estimating one.
| Model | Quant | Fits in 80 GB? | Cited speed on this card |
|---|---|---|---|
| Llama 3.1 8B 4.9 GB file | Q4_K_M | Fits~81 GB left for context | No public measurement on file |
| Qwen3 14B 9 GB file | Q4_K_M | Fits~76.9 GB left for context | No public measurement on file |
| gpt-oss-20b 21B-A3.6B · 12.1 GB file | MXFP4 | Fits~73.8 GB left for context | No public measurement on file |
| Qwen3 30B-A3B 18.6 GB file | Q4_K_M | Fits~67.3 GB left for context | No public measurement on file |
| Qwen3 32B 19.8 GB file | Q4_K_M | Fits~66.1 GB left for context | No public measurement on file |
| Gemma 3 27B 28.7 GB file | Q8_0 | Fits~57.2 GB left for context | No public measurement on file |
| Llama 3.3 70B 42.5 GB file | Q4_K_M | Fits~43.4 GB left for context | 24.3 tok/s1 cited run, ollama, per DatabaseMart |
The NVIDIA H100 PCIe 80GB is a datacenter accelerator from the Hopper family released in 2022 from NVIDIA. Key on-paper specs include 80 GB of HBM2e memory, 350W TDP. Data on this page draws on 1 synthetic benchmark result, 28 community AI inference reports (top 14 shown), 1 measured game frame-rate result, compiled from DatabaseMart, GPU-Benchmarks-on-LLM-Inference, llama.cpp GitHub Discussions, TechSpot, Markaicode, Spheron Blog; each table row links to its source. Read this page when shopping the NVIDIA H100 PCIe 80GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).
Gaming Performance (measured FPS)
Average and 1% low frame rates by game, resolution, and quality preset. Bars are scaled against the fastest result on this page.
| Game | Resolution | Settings | Relative | Avg FPS | 1% low | Source |
|---|---|---|---|---|---|---|
| Red Dead Redemption 2 | 1440p | High | 8 fps | — | TechSpot 2023-06-21 |
AI Inference Performance
Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.
| Model | Quantization | Relative | Tokens/sec | VRAM used | Source |
|---|---|---|---|---|---|
| llama2:7b | q4_0 llama.cpp | 280.7 tok/s | — | llama.cpp GitHub Discussions 2024-01 | |
| llama3:8b | Q4_K_M llama.cpp | 145.6 tok/s | — | XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-12 | |
| llama3:8b | q4_K_M llama.cpp | 144.5 tok/s | — | GPU-Benchmarks-on-LLM-Inference (GitHub, XiongjieDai) 2024-05 | |
| llama3.3:70b | FP8 vllm | 120.0 tok/s | 76.0 GB | Spheron Blog 2026-01 | |
| phi3:mini | q4_K_M ollama | 88.1 tok/s | 9.0 GB | Markaicode 2026-07-07 | |
| deepseek-r1:14b | Q4_K_M ollama | 75.0 tok/s | — | DatabaseMart 2025-02 | |
| llama3:8b | FP16 llama.cpp | 68.0 tok/s | — | XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-12 | |
| llama3:8b | FP16 llama.cpp | 67.8 tok/s | — | GPU-Benchmarks-on-LLM-Inference (GitHub, XiongjieDai) 2024-05 | |
| qwen2:72b | Q4_K_M ollama | 28.3 tok/s | — | DatabaseMart 2025-02 | |
| qwen2.5:110b | q4_K_M ollama | 20.2 tok/s | — | DatabaseMart 2026-07-08 |
Batched serving throughput
| Model | Quantization | Relative | Tokens/sec (aggregate) | VRAM used | Source |
|---|---|---|---|---|---|
| deepseek-r1-distill-qwen:7b | FP16 vllm batched | 6270.0 tok/s aggregate | 72.0 GB | DatabaseMart 2025-03 | |
| deepseek-r1-distill-llama:8b | FP16 vllm batched | 5561.0 tok/s aggregate | 72.0 GB | DatabaseMart 2025-03 | |
| deepseek-r1-distill-qwen:14b | FP16 vllm batched | 3689.2 tok/s aggregate | — | DatabaseMart 2026-08-25 | |
| deepseek-r1-distill-qwen:14b | FP16 vllm batched | 3689.0 tok/s aggregate | 72.0 GB | DatabaseMart 2025-03 |
Synthetic Benchmarks
Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.
| Benchmark | Relative | Score | Source |
|---|---|---|---|
| 3DMark Time Spy | 2,681 points | TechSpot 2023-06-21Same result reported by Appuals (citing Geekerwan) |
Full Specifications
| TDP | 350 W |
|---|---|
| VRAM | 80 GB |
| CUDA cores | 14592 |
| Memory type | HBM2e |
NVIDIA H100 PCIe 80GB — Frequently Asked Questions
What is the NVIDIA H100 PCIe 80GB best used for?
When was the NVIDIA H100 PCIe 80GB released?
Where do the NVIDIA H100 PCIe 80GB benchmark numbers come from?
Can the NVIDIA H100 PCIe 80GB run local LLMs?
Where can I buy the NVIDIA H100 PCIe 80GB?
Buying guides that rank the NVIDIA H100 PCIe 80GB's class
This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.
Editorial guides covering the NVIDIA H100 PCIe 80GB
In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.
- Tesla P40 vs RTX 3090: Used 24GB GPUs for Local LLMs in 2026
- RTX 3060 12GB Local LLM: Reddit Consensus on What Runs (2026)
- gpt-oss on a 12GB GPU: 2026 Community Consensus (RTX 3060)
- Used RTX 3090 vs New GPU for Local LLMs: 2026 Reddit Consensus
- Best GPU for Local LLMs by VRAM: 2026 Reddit Consensus
- RTX PRO 6000 vs RTX 5090 for Local 70B LLMs: 96GB vs 32GB
- gpt-oss 20B on Surface Laptop Ultra (RTX Spark) vs DGX Spark: tok/s by Config
- Dell Precision 3660 vs Lenovo ThinkStation P3 Tower for Local LLMs (2026)
- Browse all SpecPicks reviews →
More guides & deep dives from the SpecPicks archive
Browse all articles & guides →- Best Retro Handhelds in 2026: Pocket, Mid-Tier and FPGA Picks
- Emulation Hardware in 2026: FPGA, Software, and Cart-Reader Ecosystems
- How to Build a Windows 98 Retro PC in 2026
- RTX 4070 Super vs RX 7800 XT — Which to Buy in 2026
- Best 1440p Gaming GPUs in 2026
- Best Budget Gaming PC Build 2026 — ~$1,000 Parts List
- The Complete Voodoo5 5500 AGP Driver Guide (2026 Edition)
More reviews from the SpecPicks archive
Browse all reviews →- Memory Prices Climb as Samsung and SK Hynix Pour $590B Into Chips
- Samsung's 360Hz 4K QD-OLED vs Budget 4K Gaming Monitors: Is the Premium Worth It?
- Open Source AI Code Reviewers You Can Self-Host in 2025
- ZOTAC vs MSI RTX 3060 12GB: Which Twin-Fan Card Should You Buy in 2026?
- Steam Deck OLED vs Legion Go S: 2026 Comparison
- Best Budget FPS Mouse + Pad Combo: G502 Hero & QcK 2026
- How to run DeepSeek-R1 32B on Apple M4 Max
- 32GB of DDR5 Drops to $240 in New Newegg Build Bundles
- Raspberry Pi 5 16GB Ships — Is the Pi 4 8GB Still Worth It?
- 'Count Anything' AI Model Does Object Counting — Harder Than It Sounds
- Air vs AIO for the Ryzen 7 5800X: is the Noctua NH-U12S enough?
- Enthusiast Hides Gaming PC Inside Living Room Fan
- Best Sim Racing Gear for Beginners in 2026
- G4-Meromero 31B: Running the Uncensored Gemma 4 Finetune on a 12GB RTX 3060
- llama.cpp vs vLLM for Single-User Local Chat on a 12GB GPU (2026)
- Stable Diffusion WebUI Forge on an RTX 3060 12GB: Setup and Real Throughput
- Adafruit's Cyberdeck Raspberry Pi Music Workstation Guide Lands
- Best RGB Lighting for a Gaming PC Desk: LED Strips and Case Fans Done Right
- CompactFlash + IDE Storage for a Period-Correct Windows 98 Build
- Noctua NH-U12S vs ML240L RGB: Best Cooler for a 5800X
- Build a Silent Windows 98 Gaming PC With a CompactFlash IDE Drive
- Best SSD for a PS4 Pro Upgrade in 2026: 870 EVO vs BX500
- 50+ Raspberry Pi Project Ideas for 2025 (By Skill Level)
- ProgramBench: Can LLMs Rebuild Programs From Scratch?
More buying guides from SpecPicks
Browse all buying guides →- Best Gaming Monitors for 2026
- Best CPUs for Content Creators in 2026
- Best AM5 Motherboards for 2026
- Best Gaming Mice for 2026
- Best CPU Coolers for 2026
- Best CPUs for Gaming in 2026
- Best Mechanical Keyboards for Gaming in 2026
- Best External SSDs for Content Creators in 2026
- Best 1440p 240Hz Gaming Monitors in 2026
- Best PC Cases for Building in 2026
- Best 4K Monitors for Content Creators in 2026
- Best GPU for Running 27B-32B Local LLMs in 2026
- Best Controllers for PC Gaming in 2026
- Best Tools for Building and Repairing Retro PCs in 2026
- Best GPUs for 4K Gaming in 2026
- Best NVMe SSDs for Gaming in 2026
- Best Retro Gaming Consoles & Handhelds for 2026
- Best GPUs for Running Local LLMs in 2026
- Best NVMe External Enclosures for 2026
- Best Graphics Cards for Gaming in 2026
- Best DDR5 RAM for Gaming PCs in 2026
Hardware benchmark data on SpecPicks
All benchmarks →- AMD Ryzen 7 PRO 7840U — benchmarks & specs
- AMD Radeon Pro W7900 48GB — benchmarks & specs
- AMD Ryzen 5 7430U — benchmarks & specs
- AMD Ryzen 5 PRO 5645 — benchmarks & specs
- Radeon RX 7400 — benchmarks & specs
- AMD Ryzen 5 7530U — benchmarks & specs
- AMD Ryzen 7 PRO 5845 — benchmarks & specs
- Radeon RX 6650M — benchmarks & specs
- GeForce RTX 4060 Ti 8 GB — benchmarks & specs
- AMD Ryzen 9 PRO 7940HS — benchmarks & specs
- Intel Core i5-14500 — benchmarks & specs
- Ryzen 7 9800X3D — benchmarks & specs
- AMD Ryzen 7 PRO 5875U — benchmarks & specs
- Radeon RX 7700 — benchmarks & specs
- AMD Ryzen 5 9500F — benchmarks & specs
- Ryzen 5 5600X — benchmarks & specs
- AMD Ryzen 3 5300G — benchmarks & specs
- AMD Ryzen Threadripper PRO 7945WX — benchmarks & specs
- Apple M1 Max 10 Core 3200 MHz — benchmarks & specs
- Google Coral USB Accelerator — benchmarks & specs
- AMD Ryzen 3 PRO 5350GE — benchmarks & specs
- Raspberry Pi 5 16GB — benchmarks & specs
- GeForce RTX 4060 — benchmarks & specs
- ARM Apple M1 (Virtual) 4 Core 0 MHz — benchmarks & specs