001032519 001__ 1032519 001032519 005__ 20250822121304.0 001032519 0247_ $$2datacite_doi$$a10.34734/FZJ-2024-06308 001032519 037__ $$aFZJ-2024-06308 001032519 041__ $$aEnglish 001032519 1001_ $$0P:(DE-Juel1)187395$$aJohn, Chelsea Maria$$b0$$ufzj 001032519 1112_ $$aOpenGPT-X Forum$$cBerlin$$d2024-11-05 - 2024-11-05$$wGermany 001032519 245__ $$aCARAML: Systematic Evaluation of AI Workloads on Accelerators 001032519 260__ $$c2024 001032519 3367_ $$033$$2EndNote$$aConference Paper 001032519 3367_ $$2BibTeX$$aINPROCEEDINGS 001032519 3367_ $$2DRIVER$$aconferenceObject 001032519 3367_ $$2ORCID$$aCONFERENCE_POSTER 001032519 3367_ $$2DataCite$$aOutput Types/Conference Poster 001032519 3367_ $$0PUB:(DE-HGF)24$$2PUB:(DE-HGF)$$aPoster$$bposter$$mposter$$s1736144535_26243$$xOther 001032519 520__ $$aThe rapid advancement of machine learning (ML) technologies has driven the development of specialized hardware accelerators designed to facilitate more efficient model training. This paper introduces the CARAML benchmark suite, which is employed to assess performance and energy consumption during the training of transformer-based large language models and computer vision models on a range of hardware accelerators, including systems from NVIDIA, AMD, and Graphcore. CARAML provides a compact, automated, extensible, and reproducible framework for assessing the performance and energy of ML workloads across various novel hardware architectures. The design and implementation of CARAML, along with a custom power measurement tool called jpwr, are discussed in detail. 001032519 536__ $$0G:(DE-HGF)POF4-5112$$a5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0 001032519 536__ $$0G:(DE-HGF)POF4-5122$$a5122 - Future Computing & Big Data Systems (POF4-512)$$cPOF4-512$$fPOF IV$$x1 001032519 536__ $$0G:(DE-Juel-1)68GX21007F$$aOpenGPT-X - Aufbau eines Gaia-X Knotens für große KI-Sprachmodelle und innovative Sprachapplikations-Services; Teilvorhaben: Optimierung und Skalierung auf großen HPC-Systemen (68GX21007F)$$c68GX21007F$$x2 001032519 536__ $$0G:(EU-Grant)955513$$aMAELSTROM - MAchinE Learning for Scalable meTeoROlogy and cliMate (955513)$$c955513$$fH2020-JTI-EuroHPC-2019-1$$x3 001032519 536__ $$0G:(BMBF)16HPC029$$aVerbundprojekt: MAELSTROM - Skalierbarkeit von Anwendungen des Maschinellen Lernens in den Bereichen Wetter und Klimawissenschaften für das zukünftige Supercomputing (16HPC029)$$c16HPC029$$x4 001032519 536__ $$0G:(DE-Juel-1)ATML-X-DEV$$aATML-X-DEV - ATML Accelerating Devices (ATML-X-DEV)$$cATML-X-DEV$$x5 001032519 7001_ $$0P:(DE-Juel1)172888$$aNassyr, Stepan$$b1$$eCorresponding author$$ufzj 001032519 7001_ $$0P:(DE-Juel1)145478$$aHerten, Andreas$$b2$$eCorresponding author$$ufzj 001032519 7001_ $$0P:(DE-Juel1)192254$$aPenke, Carolin$$b3$$eCorresponding author$$ufzj 001032519 8564_ $$uhttps://juser.fz-juelich.de/record/1032519/files/CARAML%20Poster.pdf$$yOpenAccess 001032519 909CO $$ooai:juser.fz-juelich.de:1032519$$pec_fundedresources$$pdriver$$pVDB$$popen_access$$popenaire 001032519 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)187395$$aForschungszentrum Jülich$$b0$$kFZJ 001032519 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)172888$$aForschungszentrum Jülich$$b1$$kFZJ 001032519 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)145478$$aForschungszentrum Jülich$$b2$$kFZJ 001032519 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)192254$$aForschungszentrum Jülich$$b3$$kFZJ 001032519 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5112$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0 001032519 9131_ $$0G:(DE-HGF)POF4-512$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5122$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vSupercomputing & Big Data Infrastructures$$x1 001032519 9141_ $$y2024 001032519 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess 001032519 920__ $$lyes 001032519 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0 001032519 980__ $$aposter 001032519 980__ $$aVDB 001032519 980__ $$aUNRESTRICTED 001032519 980__ $$aI:(DE-Juel1)JSC-20090406 001032519 9801_ $$aFullTexts