Contents
Inference Engineering Distilled
A field guide to efficient, reliable model serving.
Star on GitHub11 units
- P
Preface
Why inference engineering has become a discipline of its own.
9–144 min read
- 0
Inference
Define the inference stack and decide when specialization pays off.
15–2211 min read
- 1
Prerequisites
Connect product constraints, model choice, evaluation, and serving metrics.
23–3811 min read
- 2
Models
Read model architecture through the bottlenecks it creates at inference time.
39–7012 min read
- 3
Hardware
Map compute, memory, and interconnect limits onto serving behavior.
71–9211 min read
- 4
Software
Choose runtimes and kernels with evidence from benchmarks and profiles.
93–11611 min read
- 5
Techniques
Select optimization levers by bottleneck, quality budget, and topology.
117–15212 min read
- 6
Modalities
Compare the inference pipelines and bottlenecks of text, vision, audio, and video.
153–17612 min read
- 7
Production
Operate inference as a reliable, observable, and cost-aware service.
177–20812 min read
- A
Glossary
A quick reference for the vocabulary of inference systems.
209–2306 min read
- B
Recommended Reading
A categorized route into the sources behind modern inference engineering.
231–2566 min read