AMD · GPU · CDNA 3

AMD Instinct MI300X 192GB — Benchmarks & Specs

192 GB VRAM750W TDP$15,000 MSRP2023

Short answer: the AMD Instinct MI300X 192GB generates a median 161 tokens/sec on 7-9B dense models at 4-bit (Q4) under llama.cpp, across 3 cited runs from KnightLi GPU Benchmark Scoreboard and llama.cpp GitHub. Its 192 GB of VRAM holds up to DeepSeek-R1-Distill-Llama 70B at Q4_K_M with an ~8K context, by the Will-It-Run matrix's sizing (27 of 27 models fit). A used eBay listing for it is $27,500.00, checked 2026-10-26 06:16 UTC (listings can end at any time).

Median speed 161 tok/s 3 cited runs, 7-9B at Q4, llama.cpp
Fits in 192 GB up to DeepSeek-R1-Distill-Llama 70B at Q4_K_M → at ~8K context; 27 of 27 models in the Will-It-Run matrix

View used on eBay — $27,500.00 → Search Amazon for the AMD Instinct MI300X 192GB →

Who should buy this instead: the AMD Instinct MI300X 192GB's 192 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 192 GB, start at the 128 GB unified pick.

AMD Instinct MI300X 192GB: verdict by use case

For everyday local models (7-9B dense): a median 161 tok/s at 4-bit (Q4) under llama.cpp, across 3 cited runs (KnightLi GPU Benchmark Scoreboard, llama.cpp GitHub).
For 70B-class local models (65-73B dense): a median 36.3 tok/s at 4-bit (Q4) under llama.cpp, across 5 cited runs (AMD ROCm Blog, llama.cpp GitHub).
For the largest model in 192 GB: DeepSeek-R1-Distill-Llama 70B at Q4_K_M with an ~8K context; 27 of the 27 models in the Will-It-Run matrix fit.

Each verdict quotes a figure from a linked source or a row in the tables below. Where no cited figure exists for a use case, there is no verdict line for it.

Bottom line: how fast is the AMD Instinct MI300X 192GB?

SpecPicks median: 161 tokens/sec across 3 cited single-user runs of 7-9B dense models at 4-bit (Q4) under llama.cpp (range 161–233 tok/s), from KnightLi GPU Benchmark Scoreboard, llama.cpp GitHub (2 runs). For larger models, the SpecPicks median is 36.3 tokens/sec across 5 cited single-user runs of 65-73B dense models at 4-bit (Q4) under llama.cpp (range 36.3–36.5 tok/s), from AMD ROCm Blog, llama.cpp GitHub (4 runs). One of those runs: llama3.1:70b at q4_K_M generates 36.3 tokens/sec under llama.cpp, per llama.cpp GitHub. In Geekbench 6 OpenCL it scores 379,660 points, per Geekbench Browser. It has 192 GB of VRAM, and the llama3.1:70b run above (70.6B parameters) is the largest model on file at 4-bit quantization on this card. Larger models have no sourced run on file. That run used a 512-token context; the Will-It-Run matrix sizes each model at its listed quantization with an ~8K context, and by that measure DeepSeek-R1-Distill-Llama 70B at Q4_K_M is the largest that fits in 192 GB.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the SpecPicks median is computed here from the runs it names, each a row in the AI table. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

eBay listing (Used): $27,500.00, checked 2026-10-26 06:16 UTC. eBay listings are third-party and can end at any time; as an eBay Partner Network member, SpecPicks may earn a commission.

Read the full comparison → AMD Instinct MI300X vs Consumer GPUs: What Local AI Builders Should Buy in 2026

Check what fits in 192 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the AMD Instinct MI300X 192GB's 192 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 149.5 GB free. Check a model in the Will-It-Run matrix →

What fits in the AMD Instinct MI300X 192GB's 192 GB, and what doesn't

7 of these 7 model files load whole into 192 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face. Speeds are this card's single-user generation speed from cited community runs at the Q4-class file listed; a row with no cited run says so rather than estimating one.

Model	Quant	Fits in 192 GB?	Cited speed on this card
Llama 3.1 8B 4.9 GB file	Q4_K_M	Fits~187.1 GB left for context	No public measurement on file
Qwen3 14B 9 GB file	Q4_K_M	Fits~183 GB left for context	No public measurement on file
gpt-oss-20b 21B-A3.6B · 12.1 GB file	MXFP4	Fits~179.9 GB left for context	No public measurement on file
Qwen3 30B-A3B 18.6 GB file	Q4_K_M	Fits~173.4 GB left for context	No public measurement on file
Qwen3 32B 19.8 GB file	Q4_K_M	Fits~172.2 GB left for context	No public measurement on file
Gemma 3 27B 28.7 GB file	Q8_0	Fits~163.3 GB left for context	No public measurement on file
Llama 3.3 70B 42.5 GB file	Q4_K_M	Fits~149.5 GB left for context	No public measurement on file

The AMD Instinct MI300X 192GB is a datacenter accelerator from the CDNA 3 family released in 2023 from AMD. Key on-paper specs include 192 GB of HBM3 memory, 5300 GB/s of memory bandwidth, 750W TDP. It launched with a $15,000 MSRP, though street prices typically diverge meaningfully from launch pricing. Data on this page draws on 3 synthetic benchmark results, 14 community AI inference reports (top 12 shown), compiled from llama.cpp GitHub, AMD ROCm Blog, KnightLi GPU Benchmark Scoreboard, Geekbench Browser, Medium, NextPlatform, Valohai; each table row links to its source. Read this page when shopping the AMD Instinct MI300X 192GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the AMD Instinct MI300X 192GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model	Quantization	Tokens/sec	VRAM used	Source
llama2:7b	q4_0 llama.cpp	232.9 tok/s	—	KnightLi GPU Benchmark Scoreboard 2026-04-23
deepseek-r1:8b	q4_K_M llama.cpp	161.5 tok/s	4.7 GB	llama.cpp GitHub Discussions 2025-07-12
qwen3:8b	q4_K_M llama.cpp	161.5 tok/s	—	llama.cpp GitHub 2025-07-12
llama3.1:70b	q4_K_M llama.cpp	36.5 tok/s	—	AMD ROCm Blog 2025-10
llama3:70b	q4_K_M llama.cpp	36.3 tok/s	42.5 GB	llama.cpp GitHub (Discussion #14640) 2025-07-12
llama3.1:70b	q4_K_M llama.cpp	36.3 tok/s	39.6 GB	llama.cpp GitHub (Discussion #14640) 2025-07-12
deepseek-r1-distill-llama:70b	q4_K_M llama.cpp	36.3 tok/s	—	llama.cpp GitHub 2025-07-12
deepseek-r1:70b	q4_K_M llama.cpp	36.3 tok/s	39.6 GB	llama.cpp GitHub Discussions 2025-07-12

Batched serving throughput

Aggregate output tokens per second on the AMD Instinct MI300X 192GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model	Quantization	Tokens/sec (aggregate)	VRAM used	Source
qwen1.5-moe:2.7b-a2.7b	FP16 vllm batched	2896.9 tok/s aggregate	—	Valohai 2024-10-31
llama2:70b	FP8 other batched	2530.7 tok/s aggregate	—	NextPlatform (MLPerf Inference v4.1) 2024-09-03
qwen3:8b	— vllm batched	1142.1 tok/s aggregate	—	AMD ROCm Blog 2025-07-24
llama3.1:70b	FP8 vllm batched	1131.3 tok/s aggregate	—	Medium (Sam Stoelinga) 2024-11

Synthetic Benchmarks

Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.

Synthetic benchmark scores for the AMD Instinct MI300X 192GB — higher is better. Each row links to the public database the number was taken from.
Benchmark	Score	Source
Geekbench 6 OpenCL	379,660 points	Geekbench Browser retrieved 2026-07-09Same result reported by Tom's Hardware
MLPerf Inference v4.1 — Llama-3.1-405B Server (8x MI300X)	24,110 tokens/s	AMD ROCm Blog / MLPerf 2025-01-15
MLPerf Inference v4.1 — Llama-2-70B Server (8x MI300X)	22,021 tokens/s	AMD ROCm Blog / MLPerf 2025-01-15

Full Specifications

TDP	750 W
VRAM	192 GB
Process	5 nm
Bf16 tflops	1307
Fp16 tflops	1307
Memory type	HBM3
Compute units	304
Shading units	19456
Memory bandwidth	5300 GB/s

AMD Instinct MI300X 192GB — Frequently Asked Questions

What is the AMD Instinct MI300X 192GB best used for?

AMD Instinct MI300X 192GB is a 192 GB VRAM CDNA 3-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.

When was the AMD Instinct MI300X 192GB released, and what was its launch MSRP?

AMD Instinct MI300X 192GB launched in 2023 at a $15,000 MSRP. Street prices drift from launch pricing over a product's life, so check the current listing before buying.

Where do the AMD Instinct MI300X 192GB benchmark numbers come from?

Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.

Can the AMD Instinct MI300X 192GB run local LLMs?

Yes — AI inference results for the AMD Instinct MI300X 192GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 192 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.

Where can I buy the AMD Instinct MI300X 192GB?

SpecPicks has no verified listing of the AMD Instinct MI300X 192GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the AMD Instinct MI300X 192GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Editorial guides covering the AMD Instinct MI300X 192GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →