AMD · GPU · CDNA 4

AMD Instinct MI355X 288GB — Benchmarks & Specs

288 GB VRAM1400W TDP2025

Find AMD Instinct MI355X 288GB on eBay → Search Amazon for the AMD Instinct MI355X 288GB →

Bottom line: how fast is the AMD Instinct MI355X 288GB?

In a batched serving run it reaches 103480.0 tokens/sec aggregate throughput (many concurrent requests, not one user's generation speed) running llama2:70b at MXFP4 under vllm, per AMD ROCm Blog. Its 288 GB of VRAM is the binding constraint for local inference: by capacity alone it should fit 70B-parameter open-weight models at Q4 without offloading (estimated from VRAM capacity, not a measured run).

Every measured figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the model-size range is an estimate from VRAM capacity. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

Check what fits in 288 GB: by Q4-class file size, 23 of the 23 open-weight models in the SpecPicks Will-It-Run matrix load entirely into the AMD Instinct MI355X 288GB's 288 GB with 1.5 GB left for an ~8K context, up to DeepSeek-R1-Distill-Llama 70B (a 42.5 GB Q4_K_M file per its Hugging Face listing). Check a model in the Will-It-Run matrix →

The AMD Instinct MI355X 288GB is a datacenter accelerator from the CDNA 4 family released in 2025 from AMD. Key on-paper specs include 288 GB of HBM3E memory, 8000 GB/s of memory bandwidth, 1400W TDP. Data on this page draws on 8 community AI inference reports, compiled from AMD ROCm Blog, Metrum AI, StorageReview; each table row links to its source. Read this page when shopping the AMD Instinct MI355X 288GB, comparing it against other datacenter accelerators for your server or cluster, or sizing it for a specific workload (LLM inference and serving, fine-tuning, or HPC compute).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Batched serving throughput

Aggregate output tokens per second on the AMD Instinct MI355X 288GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table.
Model	Quantization	Tokens/sec (aggregate)	VRAM used	Source
llama2:70b	MXFP4 vllm batched	103480.0 tok/s aggregate	—	AMD ROCm Blog 2026-04
gpt-oss-120b	MXFP4 vllm batched	95004.0 tok/s aggregate	—	AMD ROCm Blog 2026-04
qwen3-235b-a22b	FP8 vllm 1024 concurrent requests	14043.0 tok/s aggregate	—	Metrum AI 2026-01
llama2:70b	FP4 vllm batched	12535.0 tok/s aggregate	—	AMD ROCm Blogs (MLPerf Inference v6.0) 2026-04
llama2:70b	FP4 other batched	12535.0 tok/s aggregate	—	StorageReview 2026-04
gpt-oss-120b	FP4 vllm batched	10267.0 tok/s aggregate	—	AMD ROCm Blogs (MLPerf Inference v6.0) 2026-04
llama3.1:405b	FP4 other batched	243.0 tok/s aggregate	—	AMD ROCm Blogs 2025-09-10
kimi-k2.5	BF16 vllm 4 concurrent requests	82.7 tok/s aggregate	—	AMD ROCm Blogs (Eagle3) 2026-01

Full Specifications

TDP	1400 W
Cooling	direct liquid
VRAM	288 GB
Fp4 tflops	20100
Fp6 tflops	20100
Fp8 tflops	10100
Process	3 nm
Fp16 tflops	5000
Fp64 tflops	78.6
Memory type	HBM3E
Compute units	256
Memory bandwidth	8000 GB/s

AMD Instinct MI355X 288GB — Frequently Asked Questions

What is the AMD Instinct MI355X 288GB best used for?

AMD Instinct MI355X 288GB is a 288 GB VRAM CDNA 4-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.

When was the AMD Instinct MI355X 288GB released?

AMD Instinct MI355X 288GB was released in 2025, a CDNA 4-generation part.

Where do the AMD Instinct MI355X 288GB benchmark numbers come from?

Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.

Can the AMD Instinct MI355X 288GB run local LLMs?

Yes — AI inference results for the AMD Instinct MI355X 288GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 288 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.

Where can I buy the AMD Instinct MI355X 288GB?

SpecPicks has no verified listing of the AMD Instinct MI355X 288GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.