NVIDIA · APU · Jetson Orin

NVIDIA Jetson AGX Orin 32GB — Benchmarks & Specs

$999 MSRP2022

Short answer: the NVIDIA Jetson AGX Orin 32GB generates a median 28.0 tokens/sec on 7-9B dense models at 4-bit (Q4) under llama.cpp, across 3 cited runs from MultimodalFlow and DFRobot.

Search Amazon for the NVIDIA Jetson AGX Orin 32GB → Find the NVIDIA Jetson AGX Orin 32GB on eBay →

Bottom line: how fast is the NVIDIA Jetson AGX Orin 32GB?

SpecPicks median: 28.0 tokens/sec across 3 cited single-user runs of 7-9B dense models at 4-bit (Q4) under llama.cpp (range 1.6–31.0 tok/s), from MultimodalFlow (2 runs), DFRobot. For local LLM inference it generates 40.0 tokens/sec running llama3:8b at q4_0 under other, per DFRobot. It has 32 GB of memory shared by the CPU and GPU, and the largest models on file at 4-bit quantization on it are 8B-class: the llama3:8b run above and llama3.1:8b (q4_K_M), which generates 52.0 tokens/sec under llama.cpp, per ProventusNova. Larger models have no sourced run on file.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the memory size is the on-paper spec; the SpecPicks median is computed here from the runs it names, each a row in the AI table. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

The NVIDIA Jetson AGX Orin 32GB is an APU from the Jetson Orin family released in 2022 from NVIDIA. Key on-paper specs include 12 CPU cores, 1792 GPU cores, 32 GB of memory shared by the CPU and GPU, 204.8 GB/s of memory bandwidth. It launched with a $999 MSRP, though street prices typically diverge meaningfully from launch pricing. Data on this page draws on 16 community AI inference reports (top 13 shown), compiled from MultimodalFlow, DFRobot, IoT Digital Twin PLM, ProventusNova, arXiv:2506.09554, GitHub: dusty-nv/jetson-containers, GitHub llama.cpp discussion #17732; each table row links to its source. Read this page when shopping the NVIDIA Jetson AGX Orin 32GB, comparing it against other APUs in your build, or sizing it for a specific workload (local LLM inference).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the NVIDIA Jetson AGX Orin 32GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model	Quantization	Tokens/sec	VRAM used	Source
llama3.1:8b	q4_K_M llama.cpp	52.0 tok/s	—	ProventusNova 2026-06-10
phi3-mini:3.8b	q4_K_M llama.cpp	47.0 tok/s	2.8 GB	MultimodalFlow 2026-05-29
llama3:8b	q4_0 other	40.0 tok/s	—	DFRobot 2024-07-12
qwen3-vl:4b	q4_K_M llama.cpp	32.7 tok/s	2.3 GB	GitHub llama.cpp discussion #17732 2025-12-03
qwen2.5:7b	q4_K_M llama.cpp	31.0 tok/s	5.2 GB	MultimodalFlow 2026-05-29
llama3.1:8b	q4_K_M llama.cpp	28.0 tok/s	5.8 GB	MultimodalFlow 2026-05-29
gemma:7b	— mlc	23.0 tok/s	—	GitHub (dusty-nv/jetson-containers) 2024-05-20
gemma:2b	q4_0 other	23.0 tok/s	—	GitHub dusty-nv/jetson-containers issue #532 2024-05-20
llama2:7b	q4_0 llama.cpp	1.6 tok/s	—	DFRobot 2023-11

Batched serving throughput

Aggregate output tokens per second on the NVIDIA Jetson AGX Orin 32GB summed across many concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model	Quantization	Tokens/sec (aggregate)	VRAM used	Source
gemma2:2b	FP16 TensorRT-LLM batched	120.0 tok/s aggregate	—	IoT Digital Twin PLM 2026-04
llama3.2:3b	INT4 TensorRT-LLM batched	85.0 tok/s aggregate	3.2 GB	IoT Digital Twin PLM 2026-04
phi3.5-mini:3.8b	INT4 TensorRT-LLM batched	62.0 tok/s aggregate	—	IoT Digital Twin PLM 2026-04
phi2:2.7b	FP16 vllm batched	25.4 tok/s aggregate	5.6 GB	arXiv:2506.09554 (Understanding LLM Inferencing on Edge Accelerators) 2025-06-12

Full Specifications

Memory	32 GB
AI TOPS	275
CPU cores	12
GPU cores	1792
Memory bandwidth	204.8 GB/s

NVIDIA Jetson AGX Orin 32GB — Frequently Asked Questions

What is the NVIDIA Jetson AGX Orin 32GB best used for?

NVIDIA Jetson AGX Orin 32GB is a Jetson Orin-family integrated CPU+GPU. Mainstream desktops and gaming builds are the typical workloads.

When was the NVIDIA Jetson AGX Orin 32GB released, and what was its launch MSRP?

NVIDIA Jetson AGX Orin 32GB launched in 2022 at a $999 MSRP. Street prices drift from launch pricing over a product's life, so check the current listing before buying.

Where do the NVIDIA Jetson AGX Orin 32GB benchmark numbers come from?

Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.

How many cores does the NVIDIA Jetson AGX Orin 32GB have, and how does it benchmark?

Core and thread counts for NVIDIA Jetson AGX Orin 32GB aren't on file. Synthetic benchmarks are still being collected for this part.

Where can I buy the NVIDIA Jetson AGX Orin 32GB?

SpecPicks has no verified listing of the NVIDIA Jetson AGX Orin 32GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.