// INFERENCE
High-throughput inference
Disaggregated serving with NVIDIA Dynamo, TensorRT-LLM, and KV-cache reuse. Tokens per second per dollar, optimized.
scheduler
NVIDIA Dynamo
precision
NVFP4
technique
KV-cache + spec-decode
// FLAGSHIP
AvailableGB300 NVL72
1.1 ExaFLOPS dense FP4 per rack. The AI-factory standard.
- MEM
- 20.7 TB HBM3e / rack (72 × 288 GB)
- BW
- 8 TB/s
- NVLINK
- Gen 5
$408,000/moSpec →
// FLAGSHIP
AvailableB300
288 GB HBM3e. 15 PFLOPS dense FP4. Built for agentic AI.
- MEM
- 288 GB HBM3e
- BW
- 8 TB/s
- NVLINK
- Gen 5
$7.95/GPU-hrSpec →
// Hopper
AvailableH200 SXM
141 GB HBM3e for long-context inference without the spill.
- MEM
- 141 GB HBM3e
- BW
- 4.8 TB/s
- NVLINK
- Gen 4
$3.59/GPU-hrSpec →
TensorRT-LLMNVIDIA DynamovLLMSGLang