| Hauptseite > Publikationsdatenbank > Empowering Neuroscience Research with Portable HPC Containers |
| Poster (After Call) | FZJ-2026-05292 |
; ; ; ;
2026
This record in other databases:
Please use a persistent id in citations: doi:10.34734/FZJ-2026-05292
Abstract: Reproducibility in computational neuroscience depends not only on complex simulation packages but critically on their entire dependency graph. NEST 3.9 — a widely used neural network simulator — carries 72 transitive dependencies, including OpenMPI, Boost, jemalloc, and Python. Deploying such workflows across HPC infrastructures is hindered by site-specific dependencies, complex build environments, heterogeneous operating systems, and microarchitecture-specific compiler optimisations.<br><br>The EBRAINS Software Distribution (ESD) addresses the primary pain point for HPC users: managing conflicting dependency graphs and interoperability challenges when combining NEST with other tools in a single, verified environment. While the ESD facilitates this by rigidly tracking dependencies via Spack, synchronising identical environments from ESD cloud deployments to specific HPC host machines remains challenging, frequently resulting in dependency hell.<br><br>We present a zero-install approach using portable, HPC-enabled Apptainer container images that incorporate these complex interdependencies. Achieving near-bare-metal performance requires explicitly addressing HPC-specific hardware interfaces: large-scale distributed simulators such as NEST require the containerised MPI runtime to interface with the host network fabric, process manager, and job scheduler simultaneously. We employ a PMIx-based strategy, encapsulating OpenMPI-5.x, UCX, verbs, and a PMIx client within the container. At runtime, the containerised PMIx client connects to the host-side PMIx server exposed by SLURM, bridging container MPI to the host scheduler.<br>To support heterogeneous microarchitectures (AVX2 and AVX-512 across Intel/AMD nodes), we use QEMU user-mode emulation within Apptainer to produce architecture-optimised builds. Performance is validated through microbenchmarks and NEST scaling tests. This work will be a core aspect of the ESD’s move toward streaming containers via CernVM-FS for on-demand deployment across HPC infrastructures.
|
The record appears in these collections: |