Lecture (After Call)

FZJ-2021-05268

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Efficient Distributed GPU Programming for Exascale

Garcia de Gonzalo, S. ; Hrywniak, M.FZJ* ; Kraus, J.FZJ* ; Oden, L.FZJ* ; Herten, A. (Corresponding author)FZJ*

2021

This record in other databases:  

Please use a persistent id in citations: doi:10.5281/ZENODO.5745505

Abstract: Over the past years, GPUs became ubiquitous in HPC installations around the world. Today, they provide the majority of performance of some of the largest supercomputers (e.g. Summit, Sierra, JUWELS Booster). This trend continues in upcoming pre-exascale and exascale systems (LUMI, Leonardo; Frontier): GPUs are chosen as the core computing devices to enter this next era of HPC.To take advantage of future GPU-accelerated systems with tens of thousands of devices, application developers need to have the proper skills and tools to understand, manage, and optimize distributed GPU applications.</p> <p>In this tutorial, participants will learn techniques to efficiently program large-scale multi-GPU systems. While programming multiple GPUs with MPI is explained in detail, also advanced techniques and models (NCCL, NVSHMEM, &hellip;) are presented. Tools for analysis are used to motivate implementation of performance optimizations. The tutorial combines lectures and hands-on exercises, using Europe's fastest supercomputer, JUWELS Booster with NVIDIA A100 GPUs.

Note: Tutorial at SC21 Conference, consisting of lectures and hands-on exercises.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5121 - Supercomputing & Big Data Facilities (POF4-512) (POF4-512)
2. 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) (POF4-511)
3. ATML-X-DEV - ATML Accelerating Devices (ATML-X-DEV) (ATML-X-DEV)

Appears in the scientific report 2021

---