Contribution to a conference proceedings FZJ-2023-00122

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png
Optimizing Distributed Deep Learning in Heterogeneous Computing Platforms for Remote Sensing Data Classification

 ;  ;  ;  ;

2022
IEEE

IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala LumpurKuala Lumpur, Malaysia, 17 Jul 2022 - 22 Jul 20222022-07-172022-07-22 IEEE 2726-2729 () [10.1109/IGARSS46834.2022.9883762]

This record in other databases:  

Please use a persistent id in citations:   doi:

Abstract: Applications from Remote Sensing (RS) unveiled unique challenges to Deep Learning (DL) due to the high volume and complexity of their data. On the one hand, deep neural network architectures have the capability to automatically ex-tract informative features from RS data. On the other hand, these models have massive amounts of tunable parameters, requiring high computational capabilities. Distributed DL with data parallelism on High-Performance Computing (HPC) systems have proved necessary in dealing with the demands of DL models. Nevertheless, a single HPC system can be al-ready highly heterogeneous and include different computing resources with uneven processing power. In this context, a standard data parallelism strategy does not partition the data efficiently according to the available computing resources. This paper proposes an alternative approach to compute the gradient, which guarantees that the contribution to the gradient calculation is proportional to the processing speed of each DL model's replica. The experimental results are obtained in a heterogeneous HPC system with RS data and demonstrate that the proposed approach provides a significant training speed up and gain in the global accuracy compared to one of the state-of-the-art distributed DL framework.


Contributing Institute(s):
  1. Jülich Supercomputing Center (JSC)
Research Program(s):
  1. 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) (POF4-511)
  2. DEEP-EST - DEEP - Extreme Scale Technologies (754304) (754304)

Appears in the scientific report 2022
Database coverage:
OpenAccess
Click to display QR Code for this record

The record appears in these collections:
Document types > Events > Contributions to a conference proceedings
Workflow collections > Public records
Institute Collections > JSC
Publications database
Open Access

 Record created 2023-01-05, last modified 2023-05-02


OpenAccess:
Download fulltext PDF
External link:
Download fulltextFulltext by OpenAccess repository
Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)