A tentative lecture/assignment schedule for CS378 "Systems for ML".
| Lecture # | Date | Topic | Assignments |
| 1 | 08-26 (Mon) | Introductions and Course Overview | |
| 2 | 08-28 (Wed) | ML 101 | Setup Tutorail released |
| 08-30 (Fri) | Assignment 1 released | ||
| 09-02 (Mon) | |
||
| 3 | 09-04 (Wed) | Hardware support for ML | |
| 4 | 09-09 (Mon) | More hardware support and Distributed ML | |
| 5 | 09-11 (Wed) | Training parallelism | |
| 6 | 09-16 (Mon) | Collectives | |
| 7 | 09-18 (Wed) | Collectives and programming | Assignment 1 Due |
| 09-22 (Sun) | Assignment 2 released | ||
| 8 | 09-23 (Mon) | ML Compilation | |
| 9 | 09-25 (Wed) | Autograd | |
| 10 | 09-26 (Thu) | Guest Lecture - 9:30am, Zoom - Tianqi Chen: CMU/OctoAI - ML Compilation. | |
| 09-30 (Mon) | Quiz 1 | ||
| 11 | 10-02 (Wed) | Transformers | |
| 10-06 (Sun) | Assignment 2 due | ||
| 12 | 10-07 (Mon) | Memory optimizations | Assignment 3 released |
| 13 | 10-09 (Wed) | Inference 101 | |
| 14 | 10-11 (Fri) | Guest lecture - 4pm, Zoom - Amar Phanishayee - Meta - LLM Serving | |
| 15 | 10-14 (Mon) | Decoders and alternate transformer architectures | |
| 16 | 10-16 (Wed) | LLM Inference and Paged Attention | |
| 17 | 10-16 (Wed) | Guest lecture - 11am, Zoom - Jared Quincy Davis - Founder and CEO, Foundry | |
| 10-20 (Sun) | Assignment 3 Due | ||
| 18 | 10-21 (Mon) | LLM Fine-tuning | |
| 19 | 10-23 (Wed) | Quiz 2 Review | Assignment 4 released |
| 20 | 10-28 (Mon) | In-class guest lecture - Adi Gangidi and Karthik Ganesan - Meta - Training LLAMA 3.0 | |
| 10-30 (Wed) | Quiz 2 | ||
| 21 | 11-04 (Mon) | Mixture of Experts | |
| -- | 11-06 (Wed) | No Class | Assignment 4 due |
| 11-08 (Fri) | Assignment 5 released | ||
| 22 | 11-11 (Mon) | Flash Attention | |
| 23 | 11-13 (Wed) | Compund AI Systems and Retrieval-Augmented Generation | |
| 24 | 11-18 (Mon) | Linear Attention | |
| 25 | 11-20 (Wed) | SSM and Mamba | |
| 11-22 (Fri) | Assignment 5 due | ||
| 11-25 (Mon) | |
||
| 11-27 (Wed) | |
||
| 26 | 12-02 (Mon) | End-to-end LLM Systems | |
| 27 | 12-04 (Wed) | Research Talks | |
| 12-09 (Mon) | Research Talks | ||