The Architecture · Module 3

The Blackwell GPU

A new class of chip8 min readModule 3 of 8

At the heart of every Blackwell product is one remarkable chip. NVIDIA calls it the largest GPU ever built — and it earns the title by breaking a physical manufacturing limit and stitching two chips into one.

208B

transistors (>2.5× Hopper)

petaFLOPS peak compute

10 TB/s

die-to-die NV-HBI link

dies fused into one GPU

#Two dies, one chip

Chipmakers can only pattern a silicon area up to the reticle limit in a single exposure. Blackwell hit that ceiling — so NVIDIA built two maximum-size dies, as large as can be made today, and fused them with a 10 terabyte-per-second NV-HBI chip-to-chip link. The result behaves as one fully coherent GPU, not two — software sees a single device.

Two reticle-limit dies act as a single GPU thanks to the 10 TB/s NV-HBI bridge.

Blackwell is built on a TSMC 4NP process tailored for NVIDIA, reaching the highest compute ever on a single chip: 20 petaFLOPS. But raw FLOPS are only part of the story — it also inherits NVIDIA’s CUDA-X software ecosystem: 4 million+ developers and 3,000+ applications.

#Blackwell Ultra: tuned for reasoning

The Blackwell Ultra GPU adds even more compute and memory specifically for the age of AI reasoning. Packaged as the GB300 NVL72, it delivers 1.5× more AI performance than the GB200 NVL72 — and against the older Hopper generation the gains are dramatic:

50×

more AI reasoning productivity

35×

faster AI reasoning inference

30×

higher energy efficiency

25×

lower cost per token

All vs. NVIDIA Hopper systems. We unpack where these numbers come from in Module 6.

#The new math: low-precision number formats

A big reason Blackwell is so fast is that it does AI math in smaller numbers. The fewer bits each number uses, the more numbers the chip can push through its math units and memory at once. The work happens in Tensor Cores — specialized units for matrix multiply-and-accumulate that first appeared in the 2017 Tesla V100 and are now in their 5th generation.

The headline addition is FP4 — just 4 bits per number — including community-standardized microscaling (OCP) formats. Blackwell supports a full ladder of precisions:

Table 1 — Blackwell-supported data types, from full precision (top) to ultra-low (bottom)
Format	Bits	Typically used for
FP64	64	Scientific / HPC, maximum accuracy
FP32	32	General-purpose full precision
TF32	~19*	Faster training, near-FP32 range
FP16	16	Mixed-precision training / inference
BF16	16	Training with wider numeric range
FP8	8	Fast training & inference
INT8	8	Quantized inference (integer)
FP6	6	Efficient inference, more accuracy than FP4
FP4	4	Maximum-throughput reasoning inference

* TF32 stores in 32 bits but computes with reduced precision.

Halving the bits roughly doubles what a GPU can process — the core idea behind FP4.

The catch — and the fix

Fewer bits means less numerical precision, which can hurt accuracy. Blackwell’s 2nd-gen Transformer Engine (next module) is the clever system that keeps FP4 accurate enough to use. That’s what makes the low-precision speedups real rather than theoretical.

Continue →