Skip to main content
NVIDIA L40 48GB
NVIDIA · GPU · Ada Lovelace Pro

NVIDIA L40 48GB — Benchmarks & Specs

48 GB VRAM300W TDP$18,000 MSRP2023
Search Amazon for the NVIDIA L40 48GB → Find the NVIDIA L40 48GB on eBay →

Bottom line: how fast is the NVIDIA L40 48GB?

For local LLM inference it generates 15.3 tokens/sec running llama3.1:70b at Q4_K_M under llama.cpp, per GPU-Benchmarks-on-LLM-Inference. In Geekbench 5 OpenCL it scores 333,904 points, per technical.city. Its 48 GB of VRAM is the binding constraint for local inference: the llama3.1:70b run above (70B parameters) is the largest model on file at 4-bit quantization on this card.

Every figure above is a row in the tables below, and each row links out to the review or public benchmark database the number was taken from. SpecPicks aggregates published measurements; it does not report first-party benchmark runs.

The NVIDIA L40 48GB is a graphics card from the Ada Lovelace Pro family released in 2023 from NVIDIA. Key on-paper specs include 48 GB of GDDR6 VRAM, 300W TDP. It launched with a $18,000 MSRP, though street prices typically diverge meaningfully from launch pricing. Data on this page draws on 7 synthetic benchmark results, 11 community AI inference reports, compiled from CpuTronic, GPU-Benchmarks-on-LLM-Inference, LocalScore, technical.city, Crusoe AI Blog, Koyeb GPU Benchmarks, llama.cpp GitHub Discussion #15013, PassMark Software, vLLM GitHub Issues #5007, VMware Cloud Foundation Blog; each table row links to its source. Read this page when shopping the NVIDIA L40 48GB, comparing it against other graphics cards in your build, or sizing it for a specific workload (synthetic benchmark scores or local LLM inference).

AI Inference Performance

Tokens per second under each model + quantization. Higher = faster generation. Bars compare runs across the same model.

Local LLM inference throughput on the NVIDIA L40 48GB, in generated tokens per second for a single request (one user's generation speed). Higher is better; each row links to the community report or benchmark database it came from. Batched serving runs are listed separately below.
Model Quantization Relative Tokens/sec VRAM used Source
llama2:7b q4_0 llama.cpp 152.0 tok/s — llama.cpp GitHub Discussion #15013 2024-01-01
llama3:8b q4_K_M llama.cpp 113.6 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-05-01
llama3:8b FP16 vllm 54.1 tok/s — VMware Cloud Foundation Blog 2024-09-25
llama-3.1:8b — vllm 46.0 tok/s — Koyeb GPU Benchmarks 2024-06-01
llama3.1:8b q4_K_M llamafile 45.1 tok/s — LocalScore 2025-04-01
llama3:8b FP16 llama.cpp 43.4 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-05-01
qwen2.5:14b q4_K_M llamafile 24.7 tok/s — LocalScore 2025-04-01
llama3:70b q4_K_M llama.cpp 15.3 tok/s — XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-05-01
llama3.1:70b Q4_K_M llama.cpp 15.3 tok/s — GPU-Benchmarks-on-LLM-Inference (GitHub) 2024-07-01

Batched serving throughput

Aggregate output tokens per second on the NVIDIA L40 48GB summed across 128 concurrent requests (serving runtimes such as vLLM). This is total server throughput, not one user's generation speed — each request generates far slower — so these rows are not comparable with the single-stream table above.
Model Quantization Relative Tokens/sec (aggregate) VRAM used Source
llama3:8b W4A8KV4 QServe 128 concurrent requests 3568.8 tok/s aggregate — Crusoe AI Blog 2024-08-02
llama-2:7b FP16 vllm batched 3115.3 tok/s aggregate — vLLM GitHub Issues #5007 2024-05-23

Synthetic Benchmarks

Higher is better. Bars are scaled within each benchmark family (multi-thread, single-thread, etc.) so you can compare like-with-like at a glance.

Synthetic benchmark scores for the NVIDIA L40 48GB — higher is better. Each row links to the public database the number was taken from.
Benchmark Relative Score Source
Geekbench 5 OpenCL 333,904 points technical.city (Geekbench Browser community) 2024-01-01
Geekbench 5 OpenCL 292,357 points CpuTronic 2023-10-01
Geekbench 5 Vulkan 249,130 points CpuTronic 2023-10-01
Geekbench 5 Vulkan 237,295 points technical.city (Geekbench Browser community) 2024-01-01
PassMark G3D Mark 27,355 points PassMark Software 2025-12-11
PassMark GPU Compute 23,067 points PassMark VideoCardBenchmark 2023-07-01
Blender Benchmark 4,336 points CpuTronic 2023-10-01

Full Specifications

tdp w300
vram gb48
cuda cores18176
memory typeGDDR6

NVIDIA L40 48GB — Frequently Asked Questions

What is the NVIDIA L40 48GB best used for?
NVIDIA L40 48GB is a 48 GB VRAM Ada Lovelace Pro-family graphics card. High-end 4K gaming and local LLM inference are the headline use cases; the benchmark tables on this page place it within its generation.
When was the NVIDIA L40 48GB released, and what was its launch MSRP?
NVIDIA L40 48GB launched in 2023 at a $18,000 MSRP. Street prices drift from launch pricing over a product's life, so check the current listing before buying.
Where do the NVIDIA L40 48GB benchmark numbers come from?
Each benchmark row on this page names its source (public benchmark databases such as TechPowerUp, PassMark, Geekbench and Cinebench, and community reports such as r/LocalLLaMA threads) and links to it, so every number can be checked against the original.
Can the NVIDIA L40 48GB run local LLMs?
Yes — AI inference results for the NVIDIA L40 48GB are on file; the AI Inference Performance section on this page lists model, quantization and tokens-per-second for each run. With 48 GB VRAM, 32B-parameter open-weight models fit at Q4 quantization.
Where can I buy the NVIDIA L40 48GB?
SpecPicks has no verified listing of the NVIDIA L40 48GB on file yet; the Amazon search link at the top of this page shows its current offers. SpecPicks earns a small affiliate commission on qualifying purchases.

Buying guides that rank the NVIDIA L40 48GB's class

This page is the raw performance data. The guides below turn it into a ranked pick for a specific build.

Related GPUs

Editorial guides covering the NVIDIA L40 48GB

In-depth SpecPicks reviews, build guides, and head-to-heads referencing this graphics card.

More guides & deep dives from the SpecPicks archive

Browse all articles & guides →

More reviews from the SpecPicks archive

Browse all reviews →

More buying guides from SpecPicks

Browse all buying guides →

Hardware benchmark data on SpecPicks

All benchmarks →