Journal Article/Contribution to a conference proceedings/Contribution to a book

FZJ-2024-05813

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Analyzing HPC Monitoring Data With a View Towards Efficient Resource Utilization

Maloney, S. (Corresponding author)FZJ* ; Suarez, E.FZJ* ; Eicker, N.FZJ* ; Guimaraes, F.FZJ* ; Frings, W.FZJ*

2024
IEEE

This record in other databases:

Please use a persistent id in citations: doi:10.1109/SBAC-PAD63648.2024.00023  doi:10.34734/FZJ-2024-05813

Abstract: Compute nodes in modern HPC systems are growing in size and their hardware has become ever more diverse. Still, many HPC centers allocate the resources of full nodes exclusively to avoid contention, despite the associated risk of underutilization. This paper describes a thorough resource utilization study of CPU and GPU compute and memory capacity, and interconnect bandwidth on JUWELS, a mature leadership-class modular supercomputer, with the aim of identifying opportunities for improving utilization through advanced scheduling and node sharing. Separate analysis of CPU-only and GPU-accelerated nodes finds that CPU compute usage is already close to optimal for the CPU-only nodes, whereas there is plenty of scope for co-scheduling CPU-based jobs on GPU-accelerated nodes. Memory capacity and node-level interconnect bandwidth are sufficient to provision co-scheduled jobs. We analyze multiple one-month datasets to validate robustness of conclusions over time and compare with previous studies on other systems to establish generalizability of results.

Note: The data used for this study are available at: https://doi.org/10.26165/JUELICH-DATA/BDFBPQ 979-8-3503-5616-8/24/$31.00 © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)
2. 5122 - Future Computing & Big Data Systems (POF4-512) (POF4-512)
3. DEEP-SEA - DEEP – SOFTWARE FOR EXASCALE ARCHITECTURES (955606) (955606)
4. ATMLAO - ATML Application Optimization and User Service Tools (ATMLAO) (ATMLAO)

Appears in the scientific report 2024

---

Database coverage:

---