001007707 001__ 1007707
001007707 005__ 20250822121435.0
001007707 0247_ $$2doi$$a10.34732/XDVBLG-SVNDMJ
001007707 0247_ $$2Handle$$a2128/34532
001007707 037__ $$aFZJ-2023-02173
001007707 041__ $$aEnglish
001007707 1001_ $$0P:(DE-Juel1)187395$$aJohn, Chelsea Maria$$b0$$eCorresponding author
001007707 1112_ $$aISC High Performance 2023$$cHamburg$$d2023-05-21 - 2023-05-25$$gISC23$$wGermany
001007707 245__ $$aOpenGPT-X – Training Large Language Models on HPC Systems
001007707 260__ $$c2023
001007707 3367_ $$033$$2EndNote$$aConference Paper
001007707 3367_ $$2BibTeX$$aINPROCEEDINGS
001007707 3367_ $$2DRIVER$$aconferenceObject
001007707 3367_ $$2ORCID$$aCONFERENCE_POSTER
001007707 3367_ $$2DataCite$$aOutput Types/Conference Poster
001007707 3367_ $$0PUB:(DE-HGF)24$$2PUB:(DE-HGF)$$aPoster$$bposter$$mposter$$s1687163345_9014$$xAfter Call
001007707 520__ $$aOpenGPT-X is a German initiative to build and train large language models (LLMs). The project aims at providing an open alternative to LLMs which are up to now private property, along with a platform for researching methods to train multilingual LLMs efficiently. For that, the project not only utilizes the state-of-the-art in training models but also incorporates new methods, algorithms, and tools. Models trained within the project will be published and used for pilot language services by industry partners. In addition, further applications are expected through Gaia-X federation. LLMs can scale to more than 175 Billion parameters, which requires efficient usage of supercomputers like JUWELS Booster. Especially in the light of the recent successes of ChatGPT, our work clearly indicates that the infrastructure of supercomputing centres and initiatives aiming to provide resources to the public can have a large societal impact. This poster outlines the initial progress and future work of the project from Jülich Supercomputing Center (JSC).
001007707 536__ $$0G:(DE-HGF)POF4-5112$$a5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001007707 536__ $$0G:(DE-HGF)POF4-5121$$a5121 - Supercomputing & Big Data Facilities (POF4-512)$$cPOF4-512$$fPOF IV$$x1
001007707 536__ $$0G:(DE-Juel-1)ATML-X-DEV$$aATML-X-DEV - ATML Accelerating Devices (ATML-X-DEV)$$cATML-X-DEV$$x2
001007707 588__ $$aDataset connected to DataCite
001007707 650_7 $$2Other$$aHPC
001007707 650_7 $$2Other$$aGPU
001007707 650_7 $$2Other$$aOpenGPTX
001007707 7001_ $$0P:(DE-Juel1)187002$$aEbert, Jan$$b1$$ufzj
001007707 7001_ $$0P:(DE-Juel1)192254$$aPenke, Carolin$$b2$$ufzj
001007707 7001_ $$0P:(DE-Juel1)185654$$aKesselheim, Stefan$$b3$$ufzj
001007707 7001_ $$0P:(DE-Juel1)145478$$aHerten, Andreas$$b4$$ufzj
001007707 773__ $$a10.34732/XDVBLG-SVNDMJ
001007707 8564_ $$uhttps://juser.fz-juelich.de/record/1007707/files/ISC%202023%20OpenGPT-X%20Project%20Poster.pdf$$yOpenAccess
001007707 909CO $$ooai:juser.fz-juelich.de:1007707$$pdriver$$pVDB$$popen_access$$popenaire
001007707 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001007707 9141_ $$y2023
001007707 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)187395$$aForschungszentrum Jülich$$b0$$kFZJ
001007707 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)187002$$aForschungszentrum Jülich$$b1$$kFZJ
001007707 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)192254$$aForschungszentrum Jülich$$b2$$kFZJ
001007707 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)185654$$aForschungszentrum Jülich$$b3$$kFZJ
001007707 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)145478$$aForschungszentrum Jülich$$b4$$kFZJ
001007707 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5112$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001007707 9131_ $$0G:(DE-HGF)POF4-512$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5121$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vSupercomputing & Big Data Infrastructures$$x1
001007707 920__ $$lyes
001007707 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
001007707 980__ $$aposter
001007707 980__ $$aVDB
001007707 980__ $$aUNRESTRICTED
001007707 980__ $$aI:(DE-Juel1)JSC-20090406
001007707 980__ $$aOPENSCIENCE
001007707 9801_ $$aFullTexts