Gast ::
| Hauptseite > Publikationsdatenbank > The Effect of BERT Training on Atmospheric Data Interpolation > print |
| Hauptseite > Publikationsdatenbank > The Effect of BERT Training on Atmospheric Data Interpolation > print |
| 001 | 1045001 | ||
| 005 | 20251104202045.0 | ||
| 024 | 7 | _ | |a 10.34734/FZJ-2025-03483 |2 datacite_doi |
| 037 | _ | _ | |a FZJ-2025-03483 |
| 100 | 1 | _ | |a Semcheddine, Asma |0 P:(DE-Juel1)203330 |b 0 |e Corresponding author |
| 111 | 2 | _ | |a Dynamics Days Europe 2025 |g DDE2025 |c Thessaloniki |d 2025-06-23 - 2025-06-27 |w Greece |
| 245 | _ | _ | |a The Effect of BERT Training on Atmospheric Data Interpolation |
| 260 | _ | _ | |c 2025 |
| 336 | 7 | _ | |a Conference Paper |0 33 |2 EndNote |
| 336 | 7 | _ | |a INPROCEEDINGS |2 BibTeX |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a CONFERENCE_POSTER |2 ORCID |
| 336 | 7 | _ | |a Output Types/Conference Poster |2 DataCite |
| 336 | 7 | _ | |a Poster |b poster |m poster |0 PUB:(DE-HGF)24 |s 1762259233_12766 |2 PUB:(DE-HGF) |x After Call |
| 520 | _ | _ | |a Atmospheric science has witnessed a breakthrough in recent years by harnessing deep learning models to understand and replicate the complex relationships within and between different atmospheric variables. Atmorep [1], a foundational model of atmospheric dynamics, was developed as a task-agnostic model, trained on 40 years of hourly data in a BERT-style manner, with up to 90% of the data being masked, in order to provide a plethora of downstream applications. To further assess the model's ability to learn a comprehensive abstract representation of atmospheric data, we tested several systematic token-masking strategies (geographical masking, temporal masking, a hybrid pattern combining both, and masking along model levels) and examined their effects on its data interpolation performance. Our preliminary results indicate that the coupled-fields transformer slightly outperforms the single-field transformer, reinforcing the correlation between different atmospheric fields. At a 75% compression ratio, AtmoRep achieves good reconstruction for the temperature field and all three wind components. Additionally, AtmoRep appears to benefit from the hybrid masking pattern, offering further insights into large-scale representation learning and enhancing our understanding of data-driven atmospheric modeling. |
| 536 | _ | _ | |a 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) |0 G:(DE-HGF)POF4-5111 |c POF4-511 |f POF IV |x 0 |
| 536 | _ | _ | |a Earth System Data Exploration (ESDE) |0 G:(DE-Juel-1)ESDE |c ESDE |x 1 |
| 536 | _ | _ | |a BMFTR 01LK2316A - Warmworld Smarter (IconRep) (-01LK2316A) |0 G:(BMFTR)-01LK2316A |c -01LK2316A |x 2 |
| 700 | 1 | _ | |a Luise, Ilaria |0 P:(DE-HGF)0 |b 1 |
| 700 | 1 | _ | |a Lessig, Christian |0 P:(DE-HGF)0 |b 2 |
| 700 | 1 | _ | |a Schultz, Martin |0 P:(DE-Juel1)6952 |b 3 |
| 700 | 1 | _ | |a Langguth, Michael |0 P:(DE-Juel1)180790 |b 4 |
| 700 | 1 | _ | |a Melidonis, Savvas |0 P:(DE-Juel1)207675 |b 5 |
| 856 | 4 | _ | |u https://juser.fz-juelich.de/record/1045001/files/DDE2025_poster_asma-final3%20%281%29.pdf |y OpenAccess |
| 909 | C | O | |o oai:juser.fz-juelich.de:1045001 |p openaire |p open_access |p VDB |p driver |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)203330 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 3 |6 P:(DE-Juel1)6952 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 4 |6 P:(DE-Juel1)180790 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 5 |6 P:(DE-Juel1)207675 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action |1 G:(DE-HGF)POF4-510 |0 G:(DE-HGF)POF4-511 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Enabling Computational- & Data-Intensive Science and Engineering |9 G:(DE-HGF)POF4-5111 |x 0 |
| 914 | 1 | _ | |y 2025 |
| 915 | _ | _ | |a OpenAccess |0 StatID:(DE-HGF)0510 |2 StatID |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)JSC-20090406 |k JSC |l Jülich Supercomputing Center |x 0 |
| 980 | 1 | _ | |a FullTexts |
| 980 | _ | _ | |a poster |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a UNRESTRICTED |
| 980 | _ | _ | |a I:(DE-Juel1)JSC-20090406 |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|