001045000 001__ 1045000
001045000 005__ 20251104202045.0
001045000 037__ $$aFZJ-2025-03482
001045000 1001_ $$0P:(DE-Juel1)203330$$aSemcheddine, Asma$$b0$$eCorresponding author
001045000 1112_ $$aDynamics Days Europe 2025$$cThessaloniki$$d2025-06-23 - 2025-06-27$$gDDE2025$$wGreece
001045000 245__ $$aThe Effect of BERT Training on Atmospheric Data Interpolation
001045000 260__ $$c2025
001045000 3367_ $$0PUB:(DE-HGF)1$$2PUB:(DE-HGF)$$aAbstract$$babstract$$mabstract$$s1762265136_21397
001045000 3367_ $$033$$2EndNote$$aConference Paper
001045000 3367_ $$2BibTeX$$aINPROCEEDINGS
001045000 3367_ $$2DRIVER$$aconferenceObject
001045000 3367_ $$2DataCite$$aOutput Types/Conference Abstract
001045000 3367_ $$2ORCID$$aOTHER
001045000 520__ $$aAtmospheric science has witnessed a breakthrough in recent years by harnessing deep learning models to understand and replicate  the complex relationships within and between different atmospheric variables. Atmorep [1], a foundational model of atmospheric dynamics, was developed as a task-agnostic model, trained on 40 years of hourly data in a BERT-style manner, with up to 90% of the data being masked, in order to provide a plethora of  downstream applications. To further assess the model's ability to learn a comprehensive abstract representation of atmospheric data, we tested several systematic token-masking strategies (geographical masking, temporal masking, a hybrid pattern combining both, and masking along model levels) and examined their effects on its data interpolation performance. Our preliminary results indicate that the coupled-fields transformer slightly outperforms the single-field transformer, reinforcing the correlation between different atmospheric fields. At a 75% compression ratio, AtmoRep achieves good reconstruction for the temperature field and all three wind components. Additionally, AtmoRep appears to benefit from the hybrid masking pattern, offering further insights into large-scale representation learning and enhancing our understanding of data-driven atmospheric modeling.
001045000 536__ $$0G:(DE-HGF)POF4-5111$$a5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001045000 536__ $$0G:(DE-Juel-1)ESDE$$aEarth System Data Exploration (ESDE)$$cESDE$$x1
001045000 536__ $$0G:(BMFTR)-01LK2316A$$aBMFTR 01LK2316A - Warmworld Smarter (IconRep) (-01LK2316A)$$c-01LK2316A$$x2
001045000 7001_ $$0P:(DE-HGF)0$$aLessig, Christian$$b1
001045000 7001_ $$0P:(DE-HGF)0$$aLuise, Ilaria$$b2
001045000 7001_ $$0P:(DE-Juel1)6952$$aSchultz, Martin$$b3
001045000 7001_ $$0P:(DE-Juel1)180790$$aLangguth, Michael$$b4
001045000 7001_ $$0P:(DE-Juel1)207675$$aMelidonis, Savvas$$b5
001045000 8564_ $$uhttps://juser.fz-juelich.de/record/1045000/files/DDE2025_Asma_Atmorep_final.docx$$yRestricted
001045000 909CO $$ooai:juser.fz-juelich.de:1045000$$pVDB
001045000 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)203330$$aForschungszentrum Jülich$$b0$$kFZJ
001045000 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)6952$$aForschungszentrum Jülich$$b3$$kFZJ
001045000 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)180790$$aForschungszentrum Jülich$$b4$$kFZJ
001045000 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)207675$$aForschungszentrum Jülich$$b5$$kFZJ
001045000 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5111$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001045000 9141_ $$y2025
001045000 920__ $$lyes
001045000 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
001045000 980__ $$aabstract
001045000 980__ $$aVDB
001045000 980__ $$aI:(DE-Juel1)JSC-20090406
001045000 980__ $$aUNRESTRICTED