Home > Publications database > The Effect of BERT Training on Atmospheric Data Interpolation > print

001     1045000
005     20251104202045.0
037 _ _ |a FZJ-2025-03482
100 1 _ |a Semcheddine, Asma
|0 P:(DE-Juel1)203330
|b 0
|e Corresponding author
111 2 _ |a Dynamics Days Europe 2025
|g DDE2025
|c Thessaloniki
|d 2025-06-23 - 2025-06-27
|w Greece
245 _ _ |a The Effect of BERT Training on Atmospheric Data Interpolation
260 _ _ |c 2025
336 7 _ |a Abstract
|b abstract
|m abstract
|0 PUB:(DE-HGF)1
|s 1762265136_21397
|2 PUB:(DE-HGF)
336 7 _ |a Conference Paper
|0 33
|2 EndNote
336 7 _ |a INPROCEEDINGS
|2 BibTeX
336 7 _ |a conferenceObject
|2 DRIVER
336 7 _ |a Output Types/Conference Abstract
|2 DataCite
336 7 _ |a OTHER
|2 ORCID
520 _ _ |a Atmospheric science has witnessed a breakthrough in recent years by harnessing deep learning models to understand and replicate the complex relationships within and between different atmospheric variables. Atmorep [1], a foundational model of atmospheric dynamics, was developed as a task-agnostic model, trained on 40 years of hourly data in a BERT-style manner, with up to 90% of the data being masked, in order to provide a plethora of downstream applications. To further assess the model's ability to learn a comprehensive abstract representation of atmospheric data, we tested several systematic token-masking strategies (geographical masking, temporal masking, a hybrid pattern combining both, and masking along model levels) and examined their effects on its data interpolation performance. Our preliminary results indicate that the coupled-fields transformer slightly outperforms the single-field transformer, reinforcing the correlation between different atmospheric fields. At a 75% compression ratio, AtmoRep achieves good reconstruction for the temperature field and all three wind components. Additionally, AtmoRep appears to benefit from the hybrid masking pattern, offering further insights into large-scale representation learning and enhancing our understanding of data-driven atmospheric modeling.
536 _ _ |a 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511)
|0 G:(DE-HGF)POF4-5111
|c POF4-511
|f POF IV
|x 0
536 _ _ |a Earth System Data Exploration (ESDE)
|0 G:(DE-Juel-1)ESDE
|c ESDE
|x 1
536 _ _ |a BMFTR 01LK2316A - Warmworld Smarter (IconRep) (-01LK2316A)
|0 G:(BMFTR)-01LK2316A
|c -01LK2316A
|x 2
700 1 _ |a Lessig, Christian
|0 P:(DE-HGF)0
|b 1
700 1 _ |a Luise, Ilaria
|0 P:(DE-HGF)0
|b 2
700 1 _ |a Schultz, Martin
|0 P:(DE-Juel1)6952
|b 3
700 1 _ |a Langguth, Michael
|0 P:(DE-Juel1)180790
|b 4
700 1 _ |a Melidonis, Savvas
|0 P:(DE-Juel1)207675
|b 5
856 4 _ |u https://juser.fz-juelich.de/record/1045000/files/DDE2025_Asma_Atmorep_final.docx
|y Restricted
909 C O |o oai:juser.fz-juelich.de:1045000
|p VDB
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 0
|6 P:(DE-Juel1)203330
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 3
|6 P:(DE-Juel1)6952
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 4
|6 P:(DE-Juel1)180790
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 5
|6 P:(DE-Juel1)207675
913 1 _ |a DE-HGF
|b Key Technologies
|l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action
|1 G:(DE-HGF)POF4-510
|0 G:(DE-HGF)POF4-511
|3 G:(DE-HGF)POF4
|2 G:(DE-HGF)POF4-500
|4 G:(DE-HGF)POF
|v Enabling Computational- & Data-Intensive Science and Engineering
|9 G:(DE-HGF)POF4-5111
|x 0
914 1 _ |y 2025
920 _ _ |l yes
920 1 _ |0 I:(DE-Juel1)JSC-20090406
|k JSC
|l Jülich Supercomputing Center
|x 0
980 _ _ |a abstract
980 _ _ |a VDB
980 _ _ |a I:(DE-Juel1)JSC-20090406
980 _ _ |a UNRESTRICTED

LibraryCollectionCLSMajorCLSMinorLanguageAuthor
Marc 21