TY  - CONF
AU  - Ciangottini, Diego
AU  - Spiga, Daniele
AU  - Memon, Ahmed Shiraz
AU  - Manzi, Andrea
AU  - Filipcic, Andrej
AU  - Troja, Antonino
AU  - Fanzago, Federica
AU  - Bianchini, Giulio
AU  - Sgaravatto, Massimo
AU  - Prica, Teo
AU  - Boccali, Tommaso
AU  - Tedeschi, Tommaso
TI  - Unlocking the compute continuum: Scaling out from cloud to HPC and HTC resources
VL  - 337
SN  - 2100-014X
CY  - Les Ulis
PB  - EDP Sciences
M1  - FZJ-2026-01050
T2  - The European physical journal / Web of Conferences
SP  - 01296
PY  - 2025
AB  - In a geo-distributed computing infrastructure with heterogeneous resources (HPC and HTC and possibly cloud), a key to unlock an efficient and user-friendly access to the resources is being able to offload each specific task to the best suited location. One of the most critical problems involves the logistics of wide-area, multi-stage workflows that move back and forth between multiple resource providers. We envision a model where such a challenge can be addressed enabling a “transparent offloading” of containerized payloads using the Kubernetes API primitives creating a common cloud-native interface to access any number of external hardware machines and type of backends. Thus we created the interLink project, an open source extension to the concept of Virtual-Kubelet with a design that aims for a common abstraction over heterogeneous and distributed backends. interLink is developed by INFN in the context of interTwin, an EU funded project that aims to build a digital-twin platform (Digital Twin Engine) for sciences, and the ICSC National Research Center for High Performance Computing, Big Data and Quantum Computing in Italy. In this talk we first provide a comprehensive overview of the key features and the technical implementation. We showcase our major case studies, such as the scale-out of an analysis facility, and the distribution of ML training processes. We focus on the impacts of being able to seamlessly exploit world-class EuroHPC supercomputers with such a technology.
T2  - Conference on Computing in High Energy and Nuclear Physics
CY  - 19 Oct 2024 - 25 Oct 2024, Kraków (Poland)
Y2  - 19 Oct 2024 - 25 Oct 2024
M2  - Kraków, Poland
LB  - PUB:(DE-HGF)8 ; PUB:(DE-HGF)7
DO  - DOI:10.1051/epjconf/202533701296
UR  - https://juser.fz-juelich.de/record/1052682
ER  -