TY  - CONF
AU  - Sedona, Rocco
AU  - Cavallaro, Gabriele
AU  - Riedel, Morris
AU  - Book, Matthias
TI  - Enhancing Large Batch Size Training of Deep Models for Remote Sensing Applications
PB  - IEEE
M1  - FZJ-2021-02864
SP  - 1583-1586
PY  - 2021
AB  - A wide variety of Remote Sensing (RS) missions arecontinuously acquiring a large volume of data every day. The availability of large datasets has propelled Deep Learning (DL) methods also in the RS domain. Convolutional Neural Networks (CNNs) have become the state of the art when tackling the classification of images, however the process of training is time consuming. In this work we exploit the Layer-wise Adaptive Moments optimizer for Batch training (LAMB) optimizer to use large batch size training on High-Performance Computing (HPC) systems. With the use of LAMB combined with learning rate scheduling and warm-up strategies, the experimental results on RS data classification demonstrate that a ResNet50 can be trained faster with batch sizes up to 32K.
T2  - IEEE International Geoscience and Remote Sensing Symposium
CY  - 12 Jul 2021 - 16 Jul 2021, Brussels (Belgium)
Y2  - 12 Jul 2021 - 16 Jul 2021
M2  - Brussels, Belgium
LB  - PUB:(DE-HGF)8 ; PUB:(DE-HGF)7
UR  - <Go to ISI:>//WOS:001250139801213
DO  - DOI:10.1109/IGARSS47720.2021.9555136
UR  - https://juser.fz-juelich.de/record/893825
ER  -