TY  - CONF
AU  - Frings, Wolfgang
AU  - Ahn, Dong H.
AU  - LeGendre, Matthew
AU  - Gamblin, Todd
AU  - de Supinski, Bronis R.
AU  - Wolf, Felix
TI  - Massively parallel loading
CY  - New York, NY
PB  - ACM Press New York, New York, USA
M1  - FZJ-2014-00071
SN  - 978-1-4503-2130-3
SP  - 389-398
PY  - 2013
AB  - Dynamic linking has many advantages for managing large code bases, but dynamically linked applications have not typically scaled well on high performance computing systems. Splitting a monolithic executable into many dynamic shared object (DSO) files decreases compile time for large codes, reduces runtime memory requirements by allowing modules to be loaded and unloaded as needed, and allows common DSOs to be shared among many executables. However, launching an executable that depends on many DSOs causes a flood of file system operations at program start-up, when each process in the parallel application loads its dependencies. At large scales, this operation has an effect similar to a site-wide denial-of-service attack, as even large parallel file systems struggle to service so many simultaneous requests. In this paper, we present SPINDLE, a novel approach to parallel loading that coordinates simultaneous file system operations with a scalable network of cache server processes. Our approach is transparent to user applications. We extend the GNU loader, which is used in Linux as well as proprietary operating systems, to limit the number of simultaneous file system operations, quickly loading DSOs without thrashing the file system. Our experiments show that our prototype implementation has a low overhead and increases the scalability of Pynamic, a benchmark that stresses the dynamic loader, by a factor of 20.
T2  - the 27th international ACM conference
CY  - 10 Jun 2013 - 14 Jun 2013, Eugene (Oregon)
Y2  - 10 Jun 2013 - 14 Jun 2013
M2  - Eugene, Oregon
LB  - PUB:(DE-HGF)8 ; PUB:(DE-HGF)7
DO  - DOI:10.1145/2464996.2465020
UR  - https://juser.fz-juelich.de/record/141704
ER  -