Index for the Introduction to Scalable Deep Learning

Alexandre Strube // Jan Ebert // Stefan Kesselheim // Jenia Jitsev // Mehdi Cherti

May 8, 2023

Topics:

1.1: Access machines, slurm, etc
1.2: A Message-Passing Interface (MPI) example
2.1: Motivation, Deep Learning Basics Recap
2.2: Distributed Training and Data Parallelism
3.1: Scaling Laws and Training with Large Data
3.2: Is my code Fast? Performance Analysis
4.1: Combating Accuracy Loss in Distributed Training
4.2: Outlook on Advanced Distributed Training
5.1: Generative Adversarial Networks (GANs) Basics
5.2: Advanced GANs

Communication:

Zoom
Slack
Compute project
Course page
This document: https://go.fzj.de/scalable-dl-may-2023
Source code for this course