AMD · GPU · CDNA 4

AMD Instinct MI355X 288GB — Benchmarks & Specs

288 GB VRAM1400W TDP2025

Short answer: the AMD Instinct MI355X 288GB generates 103480 tokens/sec on llama2:70b (MXFP4) under vllm in a single cited run, per AMD ROCm Blog; there are too few comparable runs on file for a median. Its 288 GB of VRAM holds up to DeepSeek-R1-Distill-Llama 70B at Q4_K_M with an ~8K context, by the Will-It-Run matrix's sizing (27 of 27 models fit).

Single cited run 103480 tok/s llama2:70b MXFP4, per AMD ROCm Blog
Fits in 288 GB up to DeepSeek-R1-Distill-Llama 70B at Q4_K_M → at ~8K context; 27 of 27 models in the Will-It-Run matrix

Find AMD Instinct MI355X 288GB on eBay → Search Amazon for the AMD Instinct MI355X 288GB →

Who should buy this instead: the AMD Instinct MI355X 288GB's 288 GB puts it in the 48 GB+ tier of the local-LLM GPU picks by VRAM (one card per tier, each with cited tok/s). If the models you run fit in 24 GB, compare the 24 GB pick. If they need more than 288 GB, start at the 128 GB unified pick.

Bottom line: how fast is the AMD Instinct MI355X 288GB?

In a batched serving run it reaches 103480.0 tokens/sec aggregate throughput (many concurrent requests, not one user's generation speed) running llama2:70b at MXFP4 under vllm, per AMD ROCm Blog. Its 288 GB of VRAM is the binding constraint for local inference: by capacity alone it should fit 70B-parameter open-weight models at Q4 without offloading (estimated from VRAM capacity, not a measured run).

Every measured figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the model-size range is an estimate from VRAM capacity. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

Check what fits in 288 GB: by file size at each listed quantization, 27 of the 27 open-weight model files in the SpecPicks Will-It-Run matrix load entirely into the AMD Instinct MI355X 288GB's 288 GB while keeping at least 1.5 GB reserved for an ~8K context; the largest, DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing), leaves roughly 245.5 GB free. Check a model in the Will-It-Run matrix →

What fits in the AMD Instinct MI355X 288GB's 288 GB, and what doesn't

7 of these 7 model files load whole into 288 GB with room for an ~8K context. A file fits when the card keeps at least 1.5 GB free after the weights; file sizes are the GGUF downloads on Hugging Face. Speeds are this card's single-user generation speed from cited community runs at the Q4-class file listed; a row with no cited run says so rather than estimating one.

Model	Quant	Fits in 288 GB?	Cited speed on this card
Llama 3.1 8B 4.9 GB file	Q4_K_M	Fits~283.1 GB left for context	No public measurement on file
Qwen3 14B 9 GB file	Q4_K_M	Fits~279 GB left for context	No public measurement on file
gpt-oss-20b 21B-A3.6B · 12.1 GB file	MXFP4	Fits~275.9 GB left for context	No public measurement on file
Qwen3 30B-A3B 18.6 GB file	Q4_K_M	Fits~269.4 GB left for context	No public measurement on file
Qwen3 32B 19.8 GB file	Q4_K_M	Fits~268.2 GB left for context	No public measurement on file
Gemma 3 27B 28.7 GB file	Q8_0	Fits~259.3 GB left for context	No public measurement on file
Llama 3.3 70B 42.5 GB file	Q4_K_M	Fits~245.5 GB left for context	No public measurement on file

The AMD Instinct MI355X 288GB is a datacenter accelerator from the CDNA 4 family released in 2025 from AMD. Key on-paper specs include 288 GB of HBM3E memory, 8000 GB/s of memory bandwidth, 1400W TDP. Data on this page draws on 8 community AI inference reports, compiled from AMD ROCm Blog, Metrum AI, StorageReview; each table row links to its source. Read this page when shopping the AMD Instinct MI355X 288GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Batched serving throughput

Aggregate output tokens per second on the AMD Instinct MI355X 288GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table.
Model	Quantization	Tokens/sec (aggregate)	VRAM used	Source
llama2:70b	MXFP4 vllm batched	103480.0 tok/s aggregate	—	AMD ROCm Blog 2026-04
gpt-oss-120b	MXFP4 vllm batched	95004.0 tok/s aggregate	—	AMD ROCm Blog 2026-04
qwen3-235b-a22b	FP8 vllm 1024 concurrent requests	14043.0 tok/s aggregate	—	Metrum AI 2026-01
llama2:70b	FP4 vllm batched	12535.0 tok/s aggregate	—	AMD ROCm Blogs (MLPerf Inference v6.0) 2026-04
llama2:70b	FP4 other batched	12535.0 tok/s aggregate	—	StorageReview 2026-04
gpt-oss-120b	FP4 vllm batched	10267.0 tok/s aggregate	—	AMD ROCm Blogs (MLPerf Inference v6.0) 2026-04
llama3.1:405b	FP4 other batched	243.0 tok/s aggregate	—	AMD ROCm Blogs 2025-09-10
kimi-k2.5	BF16 vllm 4 concurrent requests	82.7 tok/s aggregate	—	AMD ROCm Blogs (Eagle3) 2026-01

Full Specifications

TDP	1400 W
Cooling	direct liquid
VRAM	288 GB
Fp4 tflops	20100
Fp6 tflops	20100
Fp8 tflops	10100
Process	3 nm
Fp16 tflops	5000
Fp64 tflops	78.6
Memory type	HBM3E
Compute units	256
Memory bandwidth	8000 GB/s

AMD Instinct MI355X 288GB — Frequently Asked Questions

What is the AMD Instinct MI355X 288GB best used for?

AMD Instinct MI355X 288GB is a 288 GB VRAM CDNA 4-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.

When was the AMD Instinct MI355X 288GB released?

AMD Instinct MI355X 288GB was released in 2025, a CDNA 4-generation part.

Where do the AMD Instinct MI355X 288GB benchmark numbers come from?

Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.

Can the AMD Instinct MI355X 288GB run local LLMs?

Yes — AI inference results for the AMD Instinct MI355X 288GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 288 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.

Where can I buy the AMD Instinct MI355X 288GB?

SpecPicks has no verified listing of the AMD Instinct MI355X 288GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the AMD Instinct MI355X 288GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Editorial guides covering the AMD Instinct MI355X 288GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this datacenter accelerator.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →