Home > JuOSC (Juelich Open Science Collection) > OpenGPT-X - Training Large Language Models on HPC Systems > print

001     910080
005     20250822121514.0
024 7 _ |2 Handle
|a 2128/32006
037 _ _ |a FZJ-2022-03599
041 _ _ |a English
100 1 _ |0 P:(DE-Juel1)192254
|a Penke, Carolin
|b 0
|e Corresponding author
|u fzj
111 2 _ |a 14th JLESC Workshop
|c Urbana-Champaign
|d 2022-09-28 - 2022-09-30
|w USA
245 _ _ |a OpenGPT-X - Training Large Language Models on HPC Systems
260 _ _ |c 2022
336 7 _ |0 33
|2 EndNote
|a Conference Paper
336 7 _ |2 BibTeX
|a INPROCEEDINGS
336 7 _ |2 DRIVER
|a conferenceObject
336 7 _ |2 ORCID
|a CONFERENCE_POSTER
336 7 _ |2 DataCite
|a Output Types/Conference Poster
336 7 _ |0 PUB:(DE-HGF)24
|2 PUB:(DE-HGF)
|a Poster
|b poster
|m poster
|s 1665055854_11828
|x After Call
520 _ _ |a Artificial neural networks represent an HPC workload with increasing importance. In particular the field of Natural Language Processing (NLP) has been undergoing a revolution in recent years. The training of ever larger language models, such as GPT-3, demands large HPC resources and has the potential to greatly impact everyday technology. The OpenGPT-X project was established in 2022 and aims to not leave this field to large tech companies but to provide an open, publicly funded alternative based on European values. The Jülich Supercomputing Centre is a consortium partner providing HPC infrastructure for the pre-training of the models. We research the optimization potential in the training process for example by using novel accelerator architectures.
536 _ _ |0 G:(DE-HGF)POF4-5112
|a 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)
|c POF4-511
|f POF IV
|x 0
536 _ _ |a ATML-X-DEV - ATML Accelerating Devices (ATML-X-DEV)
|0 G:(DE-Juel-1)ATML-X-DEV
|c ATML-X-DEV
|x 1
700 1 _ |0 P:(DE-Juel1)187395
|a John, Chelsea Maria
|b 1
|u fzj
700 1 _ |0 P:(DE-Juel1)145478
|a Herten, Andreas
|b 2
|u fzj
700 1 _ |0 P:(DE-Juel1)187002
|a Ebert, Jan
|b 3
|u fzj
700 1 _ |0 P:(DE-Juel1)185654
|a Kesselheim, Stefan
|b 4
|u fzj
700 1 _ |0 P:(DE-Juel1)142361
|a Suarez, Estela
|b 5
|u fzj
856 4 _ |u https://juser.fz-juelich.de/record/910080/files/OpenGPTX-Poster.pdf
|y OpenAccess
909 C O |o oai:juser.fz-juelich.de:910080
|p openaire
|p open_access
|p VDB
|p driver
910 1 _ |0 I:(DE-588b)5008462-8
|6 P:(DE-Juel1)192254
|a Forschungszentrum Jülich
|b 0
|k FZJ
910 1 _ |0 I:(DE-588b)5008462-8
|6 P:(DE-Juel1)187395
|a Forschungszentrum Jülich
|b 1
|k FZJ
910 1 _ |0 I:(DE-588b)5008462-8
|6 P:(DE-Juel1)145478
|a Forschungszentrum Jülich
|b 2
|k FZJ
910 1 _ |0 I:(DE-588b)5008462-8
|6 P:(DE-Juel1)187002
|a Forschungszentrum Jülich
|b 3
|k FZJ
910 1 _ |0 I:(DE-588b)5008462-8
|6 P:(DE-Juel1)185654
|a Forschungszentrum Jülich
|b 4
|k FZJ
910 1 _ |0 I:(DE-588b)5008462-8
|6 P:(DE-Juel1)142361
|a Forschungszentrum Jülich
|b 5
|k FZJ
913 1 _ |0 G:(DE-HGF)POF4-511
|1 G:(DE-HGF)POF4-510
|2 G:(DE-HGF)POF4-500
|3 G:(DE-HGF)POF4
|4 G:(DE-HGF)POF
|9 G:(DE-HGF)POF4-5112
|a DE-HGF
|b Key Technologies
|l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action
|v Enabling Computational- & Data-Intensive Science and Engineering
|x 0
914 1 _ |y 2022
915 _ _ |0 StatID:(DE-HGF)0510
|2 StatID
|a OpenAccess
920 1 _ |0 I:(DE-Juel1)JSC-20090406
|k JSC
|l Jülich Supercomputing Center
|x 0
980 _ _ |a OPENSCIENCE
980 1 _ |a FullTexts
980 _ _ |a poster
980 _ _ |a VDB
980 _ _ |a UNRESTRICTED
980 _ _ |a I:(DE-Juel1)JSC-20090406

LibraryCollectionCLSMajorCLSMinorLanguageAuthor
Marc 21