GB300 NVL72 Rack
Everything so far — the dual-die GPU, the engines, NVLink — comes together in NVIDIA’s flagship reasoning machine. The GB300 NVL72 is a single liquid-cooled rack that links 36 Grace CPUs and 72 Blackwell Ultra GPUs into one NVLink domain that behaves as one colossal GPU.
#What’s in the rack
The rack holds 18 compute trays and 9 NVLink Switch trays. Each compute node packs 4 Blackwell GPUs and 2 Grace CPUs. Every GPU carries 279 GB of HBM3e and delivers 15 petaFLOPS of dense FP4 — and the whole rack is liquid-cooled to pack that density in.
| Spec | GB300 NVL72 (Ultra) | GB200 NVL72 |
|---|---|---|
| GPUs | CPUs | 72 | 36 | 72 | 36 |
| Compute / NVLink-switch trays | 18 / 9 | 18 / 9 |
| FP4 Tensor Core (dense / sparse) | 1,080 / 1,440 PF | 720 / 1,440 PF |
| FP8 / FP6 (dense / sparse) | 360 / 720 PF | 360 / 720 PF |
| INT8 (dense / sparse) | 12 / 24 POPS | 360 / 720 POPS |
| FP16 / BF16 (dense / sparse) | 180 / 360 PF | 180 / 360 PF |
| FP64 Tensor Core | 100 TF | 2,880 TF |
| HBM3e memory | 20 TB | 13.5 TB |
| HBM bandwidth | 576 TB/s | up to 576 TB/s |
| NVLink bandwidth | 130 TB/s | 130 TB/s |
| CPU cores | 2,592 Arm V2 | 2,592 Arm V2 |
Notice Blackwell Ultra trades away INT8 and FP64 throughput (100 TF FP64 vs 2,880) to pour its silicon into FP4 — 1,080 vs 720 dense petaFLOPS. It’s a chip deliberately specialized for low-precision reasoning inference, not double-precision science.
#The headline gains vs Hopper
#The AI factory, and how Dynamo earns 50×
NVIDIA frames an AI factory as a trade-off between two things: total throughput per megawatt (how much the factory produces) and per-user speed (how fast each person’s answer comes). You can’t max both — the best operating points form a Pareto frontier curve, and factory output is roughly the area under it.
NVIDIA Dynamo is a real-time orchestrator that dynamically repartitions GPU, memory and NVLink resources, letting the system glide along that curve instead of being stuck at one fixed point. By tuning parallelism strategies across Blackwell Ultra GPUs, Dynamo reaches 50× the AI-factory output of Hopper while keeping latency low.
#Networking: wiring racks into a data center
One GB300 NVL72 is already a single huge computer; AI factories chain many together. Each GPU gets 800 Gb/s of throughput via a ConnectX-8 SuperNIC, over either Quantum-X800 InfiniBand or Spectrum-X Ethernet, with BlueField-3 DPUs. GB300 is the first to use PCIe Gen6 between GPU and SuperNIC — eliminating a separate PCIe switch — and the SuperNIC encrypts GPU-to-GPU traffic at line rate.
#Bonus: real-time video generation
Video is staggeringly token-heavy: a single five-second clip is about 4 million tokens (versus a 128,000-token LLM context window) and takes ~90 seconds on Hopper. Blackwell Ultra delivers a 30× speedup, enabling real-time generation from NVIDIA Cosmos world foundation models for Physical AI.
With 65× more AI FLOPS than an HGX H100 system, the GB300 NVL72 enables dramatically more inference per rack — the reason NVIDIA pitches it as a strategic necessity, not just an upgrade.