Poster (Outreach)

FZJ-2023-05811

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Heat: accelerating massive data processing in Python

Comito, C.FZJ* ; Hoppe, F. ; Götz, M. ; Gutiérrez Hermosillo Muriedas, J. P. ; Hagemeier, B.FZJ* ; Knechtges, P. ; Krajsek, K.FZJ* ; Rüttgers, A. ; Streit, A. ; Tarnawa, M.FZJ*

2023

Abstract: Manipulating and processing massive data sets is challenging. In astrophysics as in the vast majority of research communities, the standard approach involves breaking up and analyzing data in smaller chunks, a process that is both inefficient and prone to errors. The problem is exacerbated on GPUs, because of the smaller available memory.Popular solutions to distribute NumPy/SciPy computations are based on task parallelism, introducing significant runtime overhead, complicating implementation, and often limiting GPU support to one vendor.This poster illustrates an alternative based on data parallelism instead. The open-source library Heat [1, 2] builds on PyTorch and mpi4py to simplify porting of NumPy/SciPy-based code to GPU (CUDA, ROCm, including multi-GPU, multi-node clusters). Under the hood, Heat distributes massive memory-intensive operations over multi-node resources via MPI communication. From a user's perspective, Heat can be used seamlessly in the Python array ecosystem. Supported features:- distributed (multi-GPU) I/O from shared memory- easy distribution of memory-intensive operations in existing code (e.g. matrix multiplication)- interoperability within the Python array ecosystem: Heat as a backend for your massive array manipulations, statistics, signal processing, machine learning...- transparent parallelism: prototype on your laptop, run the same code on HPC cluster.I'll also touch upon Heat's current implementation roadmap, and possible paths to collaboration.[1] https://github.com/helmholtz-analytics/heat[2] M. Götz et al., "HeAT – a Distributed and GPU-accelerated Tensor Framework for Data Analytics," 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020, pp. 276-287, doi: 10.1109/BigData50022.2020.9378050.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) (POF4-511)
2. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)
3. SLNS - SimLab Neuroscience (Helmholtz-SLNS) (Helmholtz-SLNS)

Appears in the scientific report 2023

---