Skip to main content
AMD Instinct MI300X 192GB
AMD · GPU · CDNA 3

AMD Instinct MI300X 192GB — Benchmarks & Specs

192 GB VRAM750W TDP$15,000 MSRP2023
Find AMD Instinct MI300X 192GB on eBay → Search Amazon for the AMD Instinct MI300X 192GB →

Bottom line: how fast is the AMD Instinct MI300X 192GB?

SpecPicks median: 161 tokens/sec across 3 cited single-user runs of 7-9B dense models at 4-bit (Q4) under llama.cpp (range 161–233 tok/s), from KnightLi GPU Benchmark Scoreboard, llama.cpp GitHub (2 runs). For larger models, the SpecPicks median is 36.3 tokens/sec across 5 cited single-user runs of 65-73B dense models at 4-bit (Q4) under llama.cpp (range 36.3–36.5 tok/s), from AMD ROCm Blog, llama.cpp GitHub (4 runs). One of those runs: llama3.1:70b at q4_K_M generates 36.3 tokens/sec under llama.cpp, per llama.cpp GitHub. In Geekbench 6 OpenCL it scores 379,660 points, per Geekbench Browser. It has 192 GB of VRAM, and the llama3.1:70b run above (70.6B parameters) is the largest model on file at 4-bit quantization on this card. Larger models have no sourced run on file.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the SpecPicks median is computed here from the runs it names, each a row in the AI table. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

Read the full comparison → AMD Instinct MI300X vs Consumer GPUs: What Local AI Builders Should Buy in 2026

The AMD Instinct MI300X 192GB is a datacenter accelerator from the CDNA 3 family released in 2023 from AMD. Key on-paper specs include 192 GB of HBM3 memory, 5300 GB/s of memory bandwidth, 750W TDP. It launched with a $15,000 MSRP, though street prices typically diverge meaningfully from launch pricing. Data on this page draws on 4 synthetic benchmark results, 14 community AI inference reports (top 12 shown), compiled from llama.cpp GitHub, AMD ROCm Blog, KnightLi GPU Benchmark Scoreboard, Geekbench Browser, Medium, NextPlatform, Tom's Hardware, Valohai; each table row links to its source. Read this page when shopping the AMD Instinct MI300X 192GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the AMD Instinct MI300X 192GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model Quantization Relative Tokens/sec VRAM used Source
llama2:7b q4_0 llama.cpp 232.9 tok/s — KnightLi GPU Benchmark Scoreboard 2026-04-23
deepseek-r1:8b q4_K_M llama.cpp 161.5 tok/s 4.7 GB llama.cpp GitHub Discussions 2025-07-12
qwen3:8b q4_K_M llama.cpp 161.5 tok/s — llama.cpp GitHub 2025-07-12
llama3.1:70b q4_K_M llama.cpp 36.5 tok/s — AMD ROCm Blog 2025-10-01
llama3:70b q4_K_M llama.cpp 36.3 tok/s 42.5 GB llama.cpp GitHub (Discussion #14640) 2025-07-12
llama3.1:70b q4_K_M llama.cpp 36.3 tok/s 39.6 GB llama.cpp GitHub (Discussion #14640) 2025-07-12
deepseek-r1-distill-llama:70b q4_K_M llama.cpp 36.3 tok/s — llama.cpp GitHub 2025-07-12
deepseek-r1:70b q4_K_M llama.cpp 36.3 tok/s 39.6 GB llama.cpp GitHub Discussions 2025-07-12

Batched serving throughput

Aggregate output tokens per second on the AMD Instinct MI300X 192GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model Quantization Relative Tokens/sec (aggregate) VRAM used Source
qwen1.5-moe:2.7b-a2.7b FP16 vllm batched 2896.9 tok/s aggregate — Valohai 2024-10-31
llama2:70b FP8 other batched 2530.7 tok/s aggregate — NextPlatform (MLPerf Inference v4.1) 2024-09-03
qwen3:8b — vllm batched 1142.1 tok/s aggregate — AMD ROCm Blog 2025-07-24
llama3.1:70b FP8 vllm batched 1131.3 tok/s aggregate — Medium (Sam Stoelinga) 2024-11-01

Synthetic Benchmarks

Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.

Synthetic benchmark scores for the AMD Instinct MI300X 192GB — higher is better. Each row links to the public database the number was taken from.
Benchmark Relative Score Source
Geekbench 6 OpenCL 379,660 points Geekbench Browser 2024-06-17
Geekbench 6.3.0 OpenCL 379,660 points Tom's Hardware 2024-06-01
MLPerf Inference v4.1 — Llama-3.1-405B Server (8x MI300X) 24,110 tokens/s AMD ROCm Blog / MLPerf 2025-01-15
MLPerf Inference v4.1 — Llama-2-70B Server (8x MI300X) 22,021 tokens/s AMD ROCm Blog / MLPerf 2025-01-15

Full Specifications

TDP750 W
VRAM192 GB
Process5 nm
Bf16 tflops1307
Fp16 tflops1307
Memory typeHBM3
Compute units304
Shading units19456
Memory bandwidth5300 GB/s

AMD Instinct MI300X 192GB — Frequently Asked Questions

What is the AMD Instinct MI300X 192GB best used for?
AMD Instinct MI300X 192GB is a 192 GB VRAM CDNA 3-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.
When was the AMD Instinct MI300X 192GB released, and what was its launch MSRP?
AMD Instinct MI300X 192GB launched in 2023 at a $15,000 MSRP. Street prices drift from launch pricing over a product's life, so check the current listing before buying.
Where do the AMD Instinct MI300X 192GB benchmark numbers come from?
Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.
Can the AMD Instinct MI300X 192GB run local LLMs?
Yes — AI inference results for the AMD Instinct MI300X 192GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 192 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.
Where can I buy the AMD Instinct MI300X 192GB?
SpecPicks has no verified listing of the AMD Instinct MI300X 192GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the AMD Instinct MI300X 192GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Related GPUs

Editorial guides covering the AMD Instinct MI300X 192GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →