001018062 001__ 1018062
001018062 005__ 20231121201850.0
001018062 0247_ $$2doi$$a10.1145/3624062.3624259
001018062 0247_ $$2datacite_doi$$a10.34734/FZJ-2023-04518
001018062 037__ $$aFZJ-2023-04518
001018062 1001_ $$0P:(DE-Juel1)180916$$aAach, Marcel$$b0$$eCorresponding author$$ufzj
001018062 1112_ $$aSC-W 2023: Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis$$cDenver, CO$$d2023-11-12 - 2023-11-17$$gSC 2023$$wUSA
001018062 245__ $$aShort Paper: Accelerating Hyperparameter Optimization Algorithms with Mixed Precision
001018062 260__ $$bACM New York, NY, USA$$c2023
001018062 300__ $$a1776–1779
001018062 3367_ $$2ORCID$$aCONFERENCE_PAPER
001018062 3367_ $$033$$2EndNote$$aConference Paper
001018062 3367_ $$2BibTeX$$aINPROCEEDINGS
001018062 3367_ $$2DRIVER$$aconferenceObject
001018062 3367_ $$2DataCite$$aOutput Types/Conference Paper
001018062 3367_ $$0PUB:(DE-HGF)8$$2PUB:(DE-HGF)$$aContribution to a conference proceedings$$bcontrib$$mcontrib$$s1700553840_7160
001018062 520__ $$aHyperparameter Optimization (HPO) of Neural Networks (NNs) is a computationally expensive procedure. On accelerators, such as NVIDIA Graphics Processing Units (GPUs) equipped with Tensor Cores, it is possible to speed-up the NN training by reducing the precision of some of the NN parameters, also referred to as mixed precision training. This paper investigates the performance of three popular HPO algorithms in terms of the achieved speed-up and model accuracy, utilizing early stopping, Bayesian, and genetic optimization approaches, in combination with mixed precision functionalities. The benchmarks are performed on 64 GPUs in parallel on three datasets: two from the vision and one from the Computational Fluid Dynamics domain. The results show that larger speed-ups can be achieved for mixed compared to full precision HPO if the checkpoint frequency is kept low. In addition to the reduced runtime, small gains in generalization performance on the test set are observed.
001018062 536__ $$0G:(DE-HGF)POF4-5111$$a5111 - Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001018062 536__ $$0G:(EU-Grant)951733$$aRAISE - Research on AI- and Simulation-Based Engineering at Exascale (951733)$$c951733$$fH2020-INFRAEDI-2019-1$$x1
001018062 588__ $$aDataset connected to CrossRef Conference
001018062 7001_ $$0P:(DE-Juel1)188513$$aSarma, Rakesh$$b1
001018062 7001_ $$0P:(DE-Juel1)188268$$aInanc, Eray$$b2
001018062 7001_ $$0P:(DE-Juel1)132239$$aRiedel, Morris$$b3$$ufzj
001018062 7001_ $$0P:(DE-Juel1)165948$$aLintermann, Andreas$$b4
001018062 773__ $$a10.1145/3624062.3624259
001018062 8564_ $$uhttps://juser.fz-juelich.de/record/1018062/files/FZJ-2023-04518.pdf$$yOpenAccess
001018062 909CO $$ooai:juser.fz-juelich.de:1018062$$popenaire$$popen_access$$pdriver$$pVDB$$pec_fundedresources$$pdnbdelivery
001018062 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)180916$$aForschungszentrum Jülich$$b0$$kFZJ
001018062 9101_ $$0I:(DE-HGF)0$$6P:(DE-Juel1)180916$$a University of Iceland$$b0
001018062 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)188513$$aForschungszentrum Jülich$$b1$$kFZJ
001018062 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)188268$$aForschungszentrum Jülich$$b2$$kFZJ
001018062 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)132239$$aForschungszentrum Jülich$$b3$$kFZJ
001018062 9101_ $$0I:(DE-HGF)0$$6P:(DE-Juel1)132239$$a University of Iceland$$b3
001018062 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)165948$$aForschungszentrum Jülich$$b4$$kFZJ
001018062 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5111$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001018062 9141_ $$y2023
001018062 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001018062 9201_ $$0I:(DE-Juel1)JSC-20090406$$kJSC$$lJülich Supercomputing Center$$x0
001018062 980__ $$acontrib
001018062 980__ $$aVDB
001018062 980__ $$aUNRESTRICTED
001018062 980__ $$aI:(DE-Juel1)JSC-20090406
001018062 9801_ $$aFullTexts