Contribution to a conference proceedings/Contribution to a book

FZJ-2025-04254

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Performance Analysis of Inline Compression in pySDC

Lattanzio, E. (Corresponding author) ; Ranjan, S. ; Underwood, R. ; Baumann, T.FZJ* ; Speck, R.FZJ* ; Calhoun, J. C.

2025
IEEE

This record in other databases:  

Please use a persistent id in citations: doi:10.1109/HPEC67600.2025.11196695

Abstract: The volume of data required for High Performance Computing (HPC) jobs is growing faster than the memory storage available to store the required data, leading to performance bottlenecks. Hence the need for inline data compression, which reduces the amount of allocated memory needed by storing all data in its compressed format and decompressing/recompressing single variables as needed. We apply inline compression to the HPC application pySDC, a framework for parallel-in-time integration of partial differential equations. We introduce a new version of pySDC that has a compression manager to add inline compression functionality, along with a software cache that stores the decompressed state of the most frequently used variables. We use the ZFP lossy compressor to test our model with varying software cache sizes. Results show that having no cache has the best compression ratio (CR) at size 5.8, but having a cache size of 16 reduces total execution time by 2.6× while also slightly improving the memory footprint with a CR of 1.5. Our framework overall provides user versatility in the trade-off between execution time and memory savings.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)
2. RGRSE - RG Research Software Engineering for HPC (RG RSE) (RG-RSE) (RG-RSE)

Appears in the scientific report 2025

---