Harvard Opens Free Detailed Course on ML Systems Engineering

Harvard has opened free access to one of the most detailed courses on machine learning systems engineering.

CS249r goes far beyond neural network architecture. The main focus is on the engineering infrastructure that determines how models are trained and served efficiently: processors, memory hierarchy, data movement, accelerators, distributed computing, inference, quantization, serving, reliability, and deployment.

The public materials include two volumes of ML Systems, TinyTorch for building your own ML framework from scratch, interactive browser labs, MLSys·im for experimenting with hardware and infrastructure bottlenecks, StaffML for system design practice, and lecture slides.

TinyTorch contains 20 sequential modules. In the labs, you can change system parameters and immediately see where bottlenecks appear, instead of only studying theory.

The course is best taken after an introduction to neural networks and Transformers. Understanding attention is useful, but a different level of comprehension begins when you see why KV cache consumes so much memory, when inference becomes memory-bandwidth bound, how batching affects throughput, and why adding more GPUs does not always lead to proportional performance gains.

All materials are available at mlsysbook.ai.

GitHub: harvard-edge/cs249r_book