Contribution to a conference proceedings/Contribution to a book

FZJ-2021-01972

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Integration of Parallel I/O Library and Flash Native Accelerators: An Evaluation of SIONlib with IME

Chasapis, K. (Corresponding author) ; Acquaviva, J.-T. ; Lührs, S.FZJ*

2021
ACM New York, NY, USA

This record in other databases:  

Please use a persistent id in citations: http://hdl.handle.net/2128/27712  doi:10.1145/3439839.3458736

Abstract: As illustrated by the IO500 ranking performance and scalability of storage systems have improved dramatically in the recent years. Most of these progresses are due to novel hardware and architectures. The resulting solutions tend to be more complex and harder to manage from an end-user perspective. This work proposes to address simultaneously the user experience and the performance issue. The solution is based on DDN's Infinite Memory Engine (IME), an all Flash distributed cache that sits between the compute nodes and the parallel file system. IME delivers extreme performance for bulk data and complex I/O patterns but does not accelerate metadata traffic. To shield the end-user from the additional complexity of dealing with an extra storage-tier, IME accesses are embedded within SIONlib. SIONlib is a parallel I/O library, which has the ability to provide a task local like access scheme for a single file in the underlying file system that allows to reduce the number of metadata operations. This combined construction with IME and SIONlib allows end-users to obtain the Flash level of performance transparently for their applications and also to receive a performance boost on metadata operations. In respect of traditional file systems, according to the partest benchmark suite, the solution improves performance over 1.8 times on a single node and up to 4 times for multiple nodes. Moreover, experimenting with SIONlib parameters we observe that the resulting performances are more robust towards different data layout patterns and transfer sizes than with traditional file systems.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)
2. EoCoE-II - Energy Oriented Center of Excellence : toward exascale for energy (824158) (824158)

Appears in the scientific report 2021

---

Database coverage:

---