Report

FZJ-2026-00918

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Automotive Foundation Models: Fundamentals, Training Concepts and Applications

Aach, M.FZJ* ; Adolph, L. ; Baumann, S. ; Benassou, S.FZJ* ; Bernhard, D. ; Bernhard, S. ; Bouzidi, M.-K. ; Braun, M. ; Brox, T. ; Burdorf, S. ; Dauner, D. ; Graaff, T. d. ; Doğan, S. ; Derajic, B. ; Flohr, F. ; Galesso, S. ; Ganzer, M. ; Gottschalk, H. ; Heid, D. ; Heue, F. ; Hu, T. ; Hubschneider, C. ; Kästingschäfer, M. ; Keser, M. ; Khongsab, P. ; Kowol, K. ; Krause, F. ; Kromm, E. ; Lucente, G. ; Lukin, A. ; Mariani, A. ; Mittal, S. ; Mousakhan, A. ; Mualla, F. ; Molin, A. ; Neuhöfer, J. ; Niemeijer, J. ; Bernal, C. O. ; Ommer, B. ; Ourania Tze, C. ; Peyinghaus, S. ; Piecha, P. ; Prestel, U. ; Ramazzina, A. ; Reichardt, J. ; Rist, C. ; Ritter, W. ; Rochau, D. ; Savarino, F. ; Schenkel, P. ; Schlauch, C. ; Schmidt, J. ; Taş, Ö. Ş. ; Thiel, L. ; Vivekanandan, A. ; Walz, S. ; Wang, J.FZJ* ; Wang, Z. ; Wiederer, J. ; Winter, K. ; Wunderlich, C. ; Yadav, H. ; Yao, Y. ; Yang, Y.

2025

Report No.: Whitepaper 1

Abstract: Generative AI has gained wide attention through large-scale models such as DALL-E 3, StableDiffusion, and GPT-4, creating new opportunities for innovation while raising important ethicalquestions. At the core of these advancements are foundation models: massive neural networkspretrained on extensive datasets and designed to adapt across diverse tasks and data modalities.Their ability to unify camera, LiDAR, radar, and other sensor data has significant potential forenhancing perception, prediction, and planning in autonomous driving. <br>This report discusses the fundamental concepts underlying foundation models, including theirarchitectural structures, training regimens, and interpretability considerations. It examines cur-rent literature for foundation models and their application and use in the domain of autonomousdriving. Techniques for generative sensor data synthesis are outlined, demonstrating how arti-ficial datasets can replicate complex driving environments and reduce the expense of collectingreal-world samples. Methods for extending these generative approaches to sequential or videodata are also highlighted, enabling realistic motion forecasting and scenario simulation. Fi-nally, the report explores how abstract representations, such as semantic maps or symbolic data,can increase explainability and computational efficiency when applied to autonomous driving.This whitepaper aims to provide both foundational understanding and practical guidance ofthe current state-of-the-art for leveraging generative AI and foundation models in the field ofautonomous driving.

---

Contributing Institute(s):

1. Jülich Supercomputing Center (JSC)

Research Program(s):

1. 5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511) (POF4-511)
2. 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) (POF4-511)
3. nxtAIM - nxtAIM – NXT GEN AI Methods (19A23014l) (19A23014l)
4. SDL Fluids & Solids Engineering (SDLFSE)

---