Deep Learning at Scale Training

NERSC and NVIDIA are hosting a hybrid, hands-on Deep Learning at Scale training event on March 3-4 in Berkeley, CA . This training will help users explore distributed training for deep learning models on high-performance computing systems (specifically Perlmutter). The training will focus on building a large-scale deep learning model on a real scientific application (transformers for weather forecasting) and walk users through profiling tools and performance optimization on a single GPU, scaling to multiple GPUs (and nodes) through distributed training with data parallelism (along with tips and techniques to scale) as well as advanced parallelization for very large models with model parallelism. 

We will provide example code and datasets to allow attendees to experiment hands-on with optimized and scalable distributed training of our scientific deep learning model on Perlmutter. Due to the hands-on experiments on Perlmutter, the event attendance will be capped. However, all training material as well as the lecture recordings will be made available after the event. OLCF and ALCF users are welcome to attend. Training accounts will be provided if needed.

Logistics

This event will be hybrid. Onsite location (in B59, see visitor information for more details).