guest ::
| Home > Publications database > The missing link between massive data and AI: parallel computing with Heat > print |
| Home > Publications database > The missing link between massive data and AI: parallel computing with Heat > print |
| 001 | 1002257 | ||
| 005 | 20230228202230.0 | ||
| 024 | 7 | _ | |a 10.5281/ZENODO.7637978 |2 doi |
| 037 | _ | _ | |a FZJ-2023-01239 |
| 041 | _ | _ | |a English |
| 100 | 1 | _ | |a Comito, Claudia |0 P:(DE-Juel1)174573 |b 0 |e Corresponding author |
| 111 | 2 | _ | |a SciOps 2022: Artificial Intelligence for Science and Operations in Astronomy |g SciOps22 |c Garching |d 2022-05-16 - 2022-05-20 |w Germany |
| 245 | _ | _ | |a The missing link between massive data and AI: parallel computing with Heat |
| 260 | _ | _ | |c 2022 |
| 336 | 7 | _ | |a Conference Paper |0 33 |2 EndNote |
| 336 | 7 | _ | |a Other |2 DataCite |
| 336 | 7 | _ | |a INPROCEEDINGS |2 BibTeX |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a LECTURE_SPEECH |2 ORCID |
| 336 | 7 | _ | |a Conference Presentation |b conf |m conf |0 PUB:(DE-HGF)6 |s 1677573593_7385 |2 PUB:(DE-HGF) |x After Call |
| 520 | _ | _ | |a When it comes to enhancing exploitation of massive data, machine learning and AI methods are very much at the forefront of our awareness. Much less so is the need for, and complexity of, applying these techniques efficiently across memory-distributed data volumes. Heat [1, 2] is an open-source Python library for high-performance data analytics, machine learning, and deep learning. It provides highly optimized algorithms and data structures for tensor computations using CPUs, GPUs and distributed cluster systems. Heat's Numpy-like API makes writing scalable, GPU-accelerated applications straightforward - at the same time, parallelism implemented under the hood via MPI provides a significant improvement in efficiency and performance with respect to, e.g., Dask. Born out of a large-scale collaboration in applied sciences, Heat also acts a platform for collaboration and knowledge transfer within data-intensive science. In this presentation, I will show you the inner workings of the library, tell you about our collaborations with the astrophysics and space science community (massively parallel signal-processing capabilities for the SKA-MPG telescope among others) and hopefully gain from you some insight into how to best support data- intensive astro operations going forward. References: [1] Gotz, M., Debus, C., Coquelin, et al.: 'HeAT - a Distributed and GPU-accelerated Tensor Framework for Data Analytics'; [2] https://github.com/helmholtz-analytics/heat |
| 536 | _ | _ | |a 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) |0 G:(DE-HGF)POF4-5112 |c POF4-511 |f POF IV |x 0 |
| 536 | _ | _ | |a SLNS - SimLab Neuroscience (Helmholtz-SLNS) |0 G:(DE-Juel1)Helmholtz-SLNS |c Helmholtz-SLNS |x 1 |
| 588 | _ | _ | |a Dataset connected to DataCite |
| 650 | _ | 7 | |a memory-distributed computing |2 Other |
| 650 | _ | 7 | |a parallel computing |2 Other |
| 650 | _ | 7 | |a data-intensive science |2 Other |
| 650 | _ | 7 | |a Big Data Analytics |2 Other |
| 650 | _ | 7 | |a Python |2 Other |
| 650 | _ | 7 | |a Message Passing Interface |2 Other |
| 650 | _ | 7 | |a PyTorch |2 Other |
| 650 | _ | 7 | |a NumPy |2 Other |
| 650 | _ | 7 | |a machine learning |2 Other |
| 700 | 1 | _ | |a Hagemeier, Björn |0 P:(DE-Juel1)132123 |b 1 |
| 700 | 1 | _ | |a Tarnawa, Michael |0 P:(DE-Juel1)178977 |b 2 |
| 700 | 1 | _ | |a Krajsek, Kai |0 P:(DE-Juel1)129347 |b 3 |
| 773 | _ | _ | |a 10.5281/ZENODO.7637978 |
| 909 | C | O | |o oai:juser.fz-juelich.de:1002257 |p VDB |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)174573 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 1 |6 P:(DE-Juel1)132123 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 2 |6 P:(DE-Juel1)178977 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 3 |6 P:(DE-Juel1)129347 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action |1 G:(DE-HGF)POF4-510 |0 G:(DE-HGF)POF4-511 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Enabling Computational- & Data-Intensive Science and Engineering |9 G:(DE-HGF)POF4-5112 |x 0 |
| 914 | 1 | _ | |y 2022 |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)JSC-20090406 |k JSC |l Jülich Supercomputing Center |x 0 |
| 980 | _ | _ | |a conf |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a I:(DE-Juel1)JSC-20090406 |
| 980 | _ | _ | |a UNRESTRICTED |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|