NVIDIA · GPU · Ada Lovelace Pro
NVIDIA L40S 48GB — Benchmarks & Specs
48 GB VRAM350W TDP$22,000 MSRP2023
Synthetic Benchmarks
| Benchmark | Score | Unit | Source |
|---|---|---|---|
| Geekbench 6 OpenCL | 352,507 | points | Tom's Hardware |
AI Inference Performance
| Model | Quantization | Tokens/sec | Source |
|---|---|---|---|
| Llama 3 8B Instruct | W4A8KV4 (QServe) | 3568.8 | Crusoe |
| Llama 3.1 8B Instruct | FP8 | 1642.0 | Red Hat (MLPerf Inference v5.1) |
| Llama 3.1 8B Instruct | FP8 | 1207.0 | Red Hat (MLPerf Inference v5.1) |
| Llama 3.1 8B | FP16 | 325.1 | Fluence |
| Llama 3 8B | Q4_K_M | 113.6 | GPU-Benchmarks-on-LLM-Inference (GitHub) |
| Llama 3.1 8B | FP16 | 43.8 | Fluence |
| Llama 3 8B | F16 | 43.4 | GPU-Benchmarks-on-LLM-Inference (GitHub) |
| Llama 3 70B | Q4_K_M | 15.3 | GPU-Benchmarks-on-LLM-Inference (GitHub) |
Full Specifications
| tdp w | 350 |
|---|---|
| vram gb | 48 |
| cuda cores | 18176 |
| memory type | GDDR6 |