NVIDIA Jetson Orin NX 8GB Benchmarks — AI

Bottom line: how fast is the NVIDIA Jetson Orin NX 8GB?

For local LLM inference it generates 7.5 tokens/sec running gemma4:12b at q4_K_M under llama.cpp, per NVIDIA Developer Forums. Its 8 GB of memory, shared by the CPU and GPU, is the binding constraint for local inference: the gemma4:12b run above (12B parameters) is the largest model on file at 4-bit quantization on it.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from; the memory size is the on-paper spec. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

The NVIDIA Jetson Orin NX 8GB is an APU from the Jetson Orin family released in 2023 from NVIDIA. Key on-paper specs include 6 CPU cores, 512 GPU cores, 8 GB of memory shared by the CPU and GPU. It launched with a $399 MSRP, though street prices typically diverge meaningfully from launch pricing. Data on this page draws on 16 community AI inference reports (top 12 shown), compiled from NVIDIA Developer Blog, Advantech-Containers, NVIDIA Developer Forums, ProventusNova; each table row links to its source. Read this page when shopping the NVIDIA Jetson Orin NX 8GB, comparing it against other APUs in your build, or sizing it for a specific workload (local LLM inference).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the NVIDIA Jetson Orin NX 8GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from.
Model	Quantization	Tokens/sec	VRAM used	Source
smollm2:1.7b	INT4 mlc	51.5 tok/s	—	NVIDIA Technical Blog 2025-01-16
llama3.2:3b	INT4 mlc	34.5 tok/s	—	NVIDIA Technical Blog 2025-01-16
phi3.5-mini:3.8b	INT4 mlc	30.8 tok/s	—	NVIDIA Technical Blog 2025-01-16
phi-3.5-mini:3.8b	int4 mlc-llm	30.8 tok/s	—	NVIDIA Developer Blog 2025-01-01
gemma2:2b	INT4 mlc	26.6 tok/s	—	NVIDIA Technical Blog 2025-01-16
llama3.2:1b	Q8_0 llama.cpp	18.5 tok/s	1.3 GB	Advantech-Containers (GitHub) 2025-05-01
qwen2.5:7b	INT4 mlc	17.1 tok/s	—	NVIDIA Technical Blog 2025-01-16
llama3.2:1b	q4_K_M llama.cpp	16.5 tok/s	1.0 GB	Advantech-Containers (GitHub) 2025-05-01
qwen2.5:1.5b	q4_K_M llama.cpp	16.0 tok/s	1.0 GB	Advantech-Containers (GitHub) 2025-05-01
llama3.1:8b	INT4 mlc	15.9 tok/s	—	NVIDIA Technical Blog 2025-01-16
gemma4:4b	q4_K_M llama.cpp	13.0 tok/s	—	NVIDIA Developer Forums 2026-06-06
llama3.1:8b	q4_K_M llama.cpp	11.0 tok/s	—	ProventusNova 2025-01-01

Full Specifications

ram gb	8
ai tops	70
cpu cores	6
gpu cores	512

NVIDIA Jetson Orin NX 8GB — Frequently Asked Questions

What is the NVIDIA Jetson Orin NX 8GB best used for?

NVIDIA Jetson Orin NX 8GB is a Jetson Orin-family integrated CPU+GPU. Mainstream desktops and gaming builds are the typical workloads.

When was the NVIDIA Jetson Orin NX 8GB released, and what was its launch MSRP?

NVIDIA Jetson Orin NX 8GB launched in 2023 at a $399 MSRP. Street prices drift from launch pricing over a product's life, so check the current listing before buying.

Where do the NVIDIA Jetson Orin NX 8GB benchmark numbers come from?

Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.

How many cores does the NVIDIA Jetson Orin NX 8GB have, and how does it benchmark?

Core and thread counts for NVIDIA Jetson Orin NX 8GB aren't on file. Synthetic benchmarks are still being collected for this part.

Where can I buy the NVIDIA Jetson Orin NX 8GB?

SpecPicks has no verified listing of the NVIDIA Jetson Orin NX 8GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.