Skip to main content
NVIDIA A100 PCIe 80GB
NVIDIA · GPU · Ampere

NVIDIA A100 PCIe 80GB — Benchmarks & Specs

80 GB VRAM300W TDP$15,000 MSRP2021
Find NVIDIA A100 PCIe 80GB on eBay → Search Amazon for the NVIDIA A100 PCIe 80GB →

Bottom line: how fast is the NVIDIA A100 PCIe 80GB?

For local LLM inference it generates 20.1 tokens/sec running qwen2:72b at q4_K_M under ollama, per DatabaseMart. It has 80 GB of VRAM, and the largest model on file at 4-bit quantization on this card is llama3:70b (70B parameters, q4_K_M), generating 22.1 tokens/sec under llama.cpp, per XiongjieDai/GPU-Benchmarks-on-LLM-Inference. Larger models have no sourced run on file.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

The NVIDIA A100 PCIe 80GB is a datacenter accelerator from the Ampere family released in 2021 from NVIDIA. Key on-paper specs include 80 GB of HBM2e memory, 300W TDP. It launched with a $15,000 MSRP, though street prices typically diverge meaningfully from launch pricing. Data on this page draws on 19 community AI inference reports (top 12 shown), compiled from DatabaseMart, XiongjieDai/GPU-Benchmarks-on-LLM-Inference, BentoML, llama.cpp GitHub, Markaicode, Valebyte; each table row links to its source. Read this page when shopping the NVIDIA A100 PCIe 80GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the NVIDIA A100 PCIe 80GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model Quantization Relative Tokens/sec VRAM used Source
llama2:7b Q4_0 llama.cpp 200.9 tok/s — llama.cpp GitHub 2024-06-01
llama3.1:8b Q4_K_M vllm 142.0 tok/s 6.0 GB Markaicode 2024-10-01
llama3:8b q4_K_M llama.cpp 138.3 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-05-01
llama3:8b FP16 llama.cpp 54.6 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-05-01

Batched serving throughput

Aggregate output tokens per second on the NVIDIA A100 PCIe 80GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model Quantization Relative Tokens/sec (aggregate) VRAM used Source
deepseek-r1:14b FP16 vllm 50 concurrent requests 1133.6 tok/s aggregate 28.0 GB DatabaseMart 2026-08-25
llama3:70b AWQ-4bit other 100 concurrent requests 700.0 tok/s aggregate — BentoML 2024-06-05
qwq:32b FP16 vllm 50 concurrent requests 522.2 tok/s aggregate 62.0 GB DatabaseMart 2026-08-25
deepseek-r1-distill-qwen:32b BF16 vllm 50 concurrent requests 472.2 tok/s aggregate 62.0 GB Databasemart.com 2025-03-01
deepseek-r1:32b FP16 vllm 50 concurrent requests 472.2 tok/s aggregate 62.0 GB DatabaseMart 2026-08-25
gemma2:27b BF16 vllm 50 concurrent requests 302.3 tok/s aggregate 51.0 GB Databasemart.com 2025-03-01
llama3:70b AWQ-4bit vllm batched 78.0 tok/s aggregate — Valebyte 2024-08-01
deepseek-r1:14b-qwen-distill BF16 vllm 50 concurrent requests 39.5 tok/s aggregate 28.0 GB DatabaseMart GPU Benchmark Blog 2026-07-08

Full Specifications

tdp w300
vram gb80
cuda cores6912
memory typeHBM2e

NVIDIA A100 PCIe 80GB — Frequently Asked Questions

What is the NVIDIA A100 PCIe 80GB best used for?
NVIDIA A100 PCIe 80GB is a 80 GB VRAM Ampere-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.
When was the NVIDIA A100 PCIe 80GB released, and what was its launch MSRP?
NVIDIA A100 PCIe 80GB launched in 2021 at a $15,000 MSRP. Street prices drift from launch pricing over a product's life, so check the current listing before buying.
Where do the NVIDIA A100 PCIe 80GB benchmark numbers come from?
Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.
Can the NVIDIA A100 PCIe 80GB run local LLMs?
Yes — AI inference results for the NVIDIA A100 PCIe 80GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 80 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.
Where can I buy the NVIDIA A100 PCIe 80GB?
SpecPicks has no verified listing of the NVIDIA A100 PCIe 80GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the NVIDIA A100 PCIe 80GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Related GPUs

Editorial guides covering the NVIDIA A100 PCIe 80GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →