NVIDIA RTX A5000 24GB — Benchmarks & Specs

Bottom line: how fast is the NVIDIA RTX A5000 24GB?

SpecPicks median: 136 tokens/sec across 3 cited single-user runs of 7-9B dense models at 4-bit (Q4) under llama.cpp (range 130–139 tok/s), from KnightLi llama.cpp GPU Benchmark Scoreboard, llama.cpp GitHub (2 runs). At the $2,999.00 Amazon price of the cheapest new linked listing (checked 2026-11-07 11:13 UTC; prices change), that is a SpecPicks Value Index of 45.3 tok/s per $1,000. For local LLM inference it generates 26.1 tokens/sec running qwen2.5:32b at Q4_0 under ollama, per DatabaseMart. In Geekbench 5 CUDA it scores 190,987 points, per technical.city. Its 24 GB of VRAM is the binding constraint for local inference: the largest model on file at 4-bit quantization on this card is llava:34b (34B parameters, Q4_K_M), generating 27.2 tokens/sec under ollama, per DatabaseMart. That run's source does not report a context length; the Will-It-Run matrix sizes each model at its listed quantization with an ~8K context, and by that measure DeepSeek-R1-Distill-Qwen 32B at Q4_K_M is the largest that fits in 24 GB.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the SpecPicks median is computed here from the runs it names, each a row in the AI table, and the per-$1,000 figure divides it by the dated listing price. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.