001     1019996
005     20240226075235.0
037 _ _ |a FZJ-2023-05811
100 1 _ |a Comito, Claudia
|0 P:(DE-Juel1)174573
|b 0
|u fzj
111 2 _ |a CS & Physics Meet-Up by Lamarr & B3D
|c TU Dortmund
|d 2023-11-29 - 2023-12-01
|w Germany
245 _ _ |a Heat: accelerating massive data processing in Python
260 _ _ |c 2023
336 7 _ |a Conference Paper
|0 33
|2 EndNote
336 7 _ |a INPROCEEDINGS
|2 BibTeX
336 7 _ |a conferenceObject
|2 DRIVER
336 7 _ |a CONFERENCE_POSTER
|2 ORCID
336 7 _ |a Output Types/Conference Poster
|2 DataCite
336 7 _ |a Poster
|b poster
|m poster
|0 PUB:(DE-HGF)24
|s 1704433621_29017
|2 PUB:(DE-HGF)
|x Outreach
520 _ _ |a Manipulating and processing massive data sets is challenging. In astrophysics as in the vast majority of research communities, the standard approach involves breaking up and analyzing data in smaller chunks, a process that is both inefficient and prone to errors. The problem is exacerbated on GPUs, because of the smaller available memory.Popular solutions to distribute NumPy/SciPy computations are based on task parallelism, introducing significant runtime overhead, complicating implementation, and often limiting GPU support to one vendor.This poster illustrates an alternative based on data parallelism instead. The open-source library Heat [1, 2] builds on PyTorch and mpi4py to simplify porting of NumPy/SciPy-based code to GPU (CUDA, ROCm, including multi-GPU, multi-node clusters). Under the hood, Heat distributes massive memory-intensive operations over multi-node resources via MPI communication. From a user's perspective, Heat can be used seamlessly in the Python array ecosystem. Supported features:- distributed (multi-GPU) I/O from shared memory- easy distribution of memory-intensive operations in existing code (e.g. matrix multiplication)- interoperability within the Python array ecosystem: Heat as a backend for your massive array manipulations, statistics, signal processing, machine learning...- transparent parallelism: prototype on your laptop, run the same code on HPC cluster.I'll also touch upon Heat's current implementation roadmap, and possible paths to collaboration.[1] https://github.com/helmholtz-analytics/heat[2] M. Götz et al., "HeAT – a Distributed and GPU-accelerated Tensor Framework for Data Analytics," 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020, pp. 276-287, doi: 10.1109/BigData50022.2020.9378050.
536 _ _ |a 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511)
|0 G:(DE-HGF)POF4-5111
|c POF4-511
|f POF IV
|x 0
536 _ _ |a 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)
|0 G:(DE-HGF)POF4-5112
|c POF4-511
|f POF IV
|x 1
536 _ _ |a SLNS - SimLab Neuroscience (Helmholtz-SLNS)
|0 G:(DE-Juel1)Helmholtz-SLNS
|c Helmholtz-SLNS
|x 2
700 1 _ |a Hoppe, Fabian
|0 P:(DE-HGF)0
|b 1
700 1 _ |a Götz, Markus
|0 P:(DE-HGF)0
|b 2
700 1 _ |a Gutiérrez Hermosillo Muriedas, Juan Pedro
|0 P:(DE-HGF)0
|b 3
700 1 _ |a Hagemeier, Björn
|0 P:(DE-Juel1)132123
|b 4
|u fzj
700 1 _ |a Knechtges, Philipp
|0 P:(DE-HGF)0
|b 5
700 1 _ |a Krajsek, Kai
|0 P:(DE-Juel1)129347
|b 6
|u fzj
700 1 _ |a Rüttgers, Alexander
|0 P:(DE-HGF)0
|b 7
700 1 _ |a Streit, Achim
|0 P:(DE-HGF)0
|b 8
700 1 _ |a Tarnawa, Michael
|0 P:(DE-Juel1)178977
|b 9
|u fzj
909 C O |o oai:juser.fz-juelich.de:1019996
|p VDB
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 0
|6 P:(DE-Juel1)174573
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 4
|6 P:(DE-Juel1)132123
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 6
|6 P:(DE-Juel1)129347
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 9
|6 P:(DE-Juel1)178977
913 1 _ |a DE-HGF
|b Key Technologies
|l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action
|1 G:(DE-HGF)POF4-510
|0 G:(DE-HGF)POF4-511
|3 G:(DE-HGF)POF4
|2 G:(DE-HGF)POF4-500
|4 G:(DE-HGF)POF
|v Enabling Computational- & Data-Intensive Science and Engineering
|9 G:(DE-HGF)POF4-5111
|x 0
913 1 _ |a DE-HGF
|b Key Technologies
|l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action
|1 G:(DE-HGF)POF4-510
|0 G:(DE-HGF)POF4-511
|3 G:(DE-HGF)POF4
|2 G:(DE-HGF)POF4-500
|4 G:(DE-HGF)POF
|v Enabling Computational- & Data-Intensive Science and Engineering
|9 G:(DE-HGF)POF4-5112
|x 1
914 1 _ |y 2023
920 1 _ |0 I:(DE-Juel1)JSC-20090406
|k JSC
|l Jülich Supercomputing Center
|x 0
980 _ _ |a poster
980 _ _ |a VDB
980 _ _ |a I:(DE-Juel1)JSC-20090406
980 _ _ |a UNRESTRICTED


LibraryCollectionCLSMajorCLSMinorLanguageAuthor
Marc 21