Name: [ONLINE] Introduction to Scalable Deep Learning @ JSC
Start: 2022-03-14T09:00:00+01:00
End: 2022-03-18T13:15:00+01:00
Location: No location set

Monday, March 14, 2022
9:00 AM Introduction to Supercomputers
Introduction to Supercomputers
9:00 AM - 11:00 AM
Room: Online
Contributions

9:00 AM Introduction

9:30 AM Lecture 1.1: Intro to supercomputers

10:00 AM Tutorial 1.1: First Steps on the Supercomputer
11:30 AM Introduction to parallel programming
Introduction to parallel programming
11:30 AM - 1:00 PM
Room: Online
Contributions

11:30 AM Lecture 1.2: Supercomputer architecture and MPI primer

12:00 PM Tutorial 1.2: Hello MPI World
Tuesday, March 15, 2022
9:30 AM Deep Learning Basics Recap
Deep Learning Basics Recap
9:30 AM - 11:00 AM
Room: Online
Contributions

9:30 AM Lecture 2.1: Motivation for Large Scale Deep Learning and Deep Learning Basics Recap

10:00 AM Tutorial 2.1: Deep Learning Basics Recap
11:30 AM Distributed Training
Distributed Training
11:30 AM - 1:00 PM
Room: Online
Contributions

11:30 AM Lecture 2.2: Distributed Training Schemes and Data Parallelism with Horovod

12:00 PM Tutorial 2.2: A first parallelization with Horovod
Wednesday, March 16, 2022
9:30 AM Large Datasets and Scaling Distributed Training
Large Datasets and Scaling Distributed Training
9:30 AM - 11:00 AM
Room: Online
Contributions

9:30 AM Lecture 3.1: Large Datasets and Scaling Distributed Training

10:00 AM Tutorial 3.1: Distributed Training with ImageNet and Scaling Basics
11:30 AM Performance analysis
Performance analysis
11:30 AM - 1:00 PM
Room: Online
Contributions

11:30 AM Lecture 3.2: Is my code fast? Performance analysis

12:00 PM Tutorial 3.2: Tools for training performance analysis
Thursday, March 17, 2022
9:30 AM Combating Accuracy Loss
Combating Accuracy Loss
9:30 AM - 11:00 AM
Room: Online
Contributions

9:30 AM Lecture 4.1: Combating Accuracy Loss

10:00 AM Tutorial 4.1 Part I : Combating Accuracy Loss, Basics
11:30 AM Outllook
Outllook
11:30 AM - 1:00 PM
Room: Online
Contributions

11:30 AM Tutorial 4.1 Part II : Combating Accuracy Loss, Advanced

12:30 PM Lecture 4.2 : Outlook beyond the basics
Friday, March 18, 2022
9:30 AM GANs I
GANs I
9:30 AM - 11:00 AM
Room: Online
Contributions

9:30 AM Lecture 5.1: Generative models, GAN basics

10:00 AM Tutorial 5.1: Parallelizing a basic GAN: DCGAN architecture
11:30 AM GANs II
GANs II
11:30 AM - 1:15 PM
Room: Online
Contributions

11:30 AM Lecture 5.2: Advanced GANs

12:00 PM Tutorial 5.2: Parallelizing an advanced GAN: StyleGAN2 architecture

1:00 PM Concluding Remarks