001     1021206
005     20250903202255.0
024 7 _ |a 10.34734/FZJ-2024-00647
|2 datacite_doi
037 _ _ |a FZJ-2024-00647
041 _ _ |a English
100 1 _ |a Quercia, Alessio
|0 P:(DE-Juel1)188471
|b 0
|e Corresponding author
|u fzj
111 2 _ |a International Conference on Data Mining
|g ICDM2023
|c Shanghai
|d 2023-12-01 - 2023-12-04
|w Peoples R China
245 _ _ |a SGD Biased towards Early Important Samples for Efficient Training
260 _ _ |c 2023
336 7 _ |a Conference Paper
|0 33
|2 EndNote
336 7 _ |a Other
|2 DataCite
336 7 _ |a INPROCEEDINGS
|2 BibTeX
336 7 _ |a conferenceObject
|2 DRIVER
336 7 _ |a LECTURE_SPEECH
|2 ORCID
336 7 _ |a Conference Presentation
|b conf
|m conf
|0 PUB:(DE-HGF)6
|s 1756892510_26549
|2 PUB:(DE-HGF)
|x After Call
520 _ _ |a In deep learning, using larger training datasets usually leads to more accurate models. However, simply adding more but redundant data may be inefficient, as some training samples may be more informative than others. We propose to bias SGD (Stochastic Gradient Descent) towards samples that are found to be more important after a few training epochs, by sampling them more often for the rest of training. In contrast to state-of-the-art, our approach requires less computational overhead to estimate sample importance, as it computes estimates once during training using the prediction probabilities, and does not require that training be restarted. In the experimental evaluation, we see that our learning technique trains faster than state-of-the-art and can achieve higher test accuracy, especially when datasets are not well balanced. Lastly, results suggest that our approach has intrinsic balancing properties. Code is available at https://github.com/AlessioQuercia/sgd biased.
536 _ _ |a 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)
|0 G:(DE-HGF)POF4-5112
|c POF4-511
|f POF IV
|x 0
536 _ _ |a HDS LEE - Helmholtz School for Data Science in Life, Earth and Energy (HDS LEE) (HDS-LEE-20190612)
|0 G:(DE-Juel1)HDS-LEE-20190612
|c HDS-LEE-20190612
|x 1
700 1 _ |a Morrison, Abigail
|0 P:(DE-Juel1)151166
|b 1
|u fzj
700 1 _ |a Scharr, Hanno
|0 P:(DE-Juel1)129394
|b 2
|u fzj
700 1 _ |a Assent, Ira
|0 P:(DE-Juel1)188313
|b 3
|u fzj
856 4 _ |u https://juser.fz-juelich.de/record/1021206/files/SGD%20Biased%20towards%20Early%20Important%20Samples%20for%20Efficient%20Training.pdf
|y OpenAccess
856 4 _ |u https://juser.fz-juelich.de/record/1021206/files/SGD%20Biased%20towards%20Early%20Important%20Samples%20for%20Efficient%20Training.gif?subformat=icon
|x icon
|y OpenAccess
856 4 _ |u https://juser.fz-juelich.de/record/1021206/files/SGD%20Biased%20towards%20Early%20Important%20Samples%20for%20Efficient%20Training.jpg?subformat=icon-1440
|x icon-1440
|y OpenAccess
856 4 _ |u https://juser.fz-juelich.de/record/1021206/files/SGD%20Biased%20towards%20Early%20Important%20Samples%20for%20Efficient%20Training.jpg?subformat=icon-180
|x icon-180
|y OpenAccess
856 4 _ |u https://juser.fz-juelich.de/record/1021206/files/SGD%20Biased%20towards%20Early%20Important%20Samples%20for%20Efficient%20Training.jpg?subformat=icon-640
|x icon-640
|y OpenAccess
909 C O |o oai:juser.fz-juelich.de:1021206
|p openaire
|p open_access
|p VDB
|p driver
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 0
|6 P:(DE-Juel1)188471
910 1 _ |a RWTH Aachen
|0 I:(DE-588b)36225-6
|k RWTH
|b 0
|6 P:(DE-Juel1)188471
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 1
|6 P:(DE-Juel1)151166
910 1 _ |a RWTH Aachen
|0 I:(DE-588b)36225-6
|k RWTH
|b 1
|6 P:(DE-Juel1)151166
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 2
|6 P:(DE-Juel1)129394
910 1 _ |a Forschungszentrum Jülich
|0 I:(DE-588b)5008462-8
|k FZJ
|b 3
|6 P:(DE-Juel1)188313
913 1 _ |a DE-HGF
|b Key Technologies
|l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action
|1 G:(DE-HGF)POF4-510
|0 G:(DE-HGF)POF4-511
|3 G:(DE-HGF)POF4
|2 G:(DE-HGF)POF4-500
|4 G:(DE-HGF)POF
|v Enabling Computational- & Data-Intensive Science and Engineering
|9 G:(DE-HGF)POF4-5112
|x 0
914 1 _ |y 2023
915 _ _ |a OpenAccess
|0 StatID:(DE-HGF)0510
|2 StatID
920 _ _ |l yes
920 1 _ |0 I:(DE-Juel1)IAS-8-20210421
|k IAS-8
|l Datenanalyse und Maschinenlernen
|x 0
920 1 _ |0 I:(DE-Juel1)IAS-6-20130828
|k IAS-6
|l Computational and Systems Neuroscience
|x 1
980 _ _ |a conf
980 _ _ |a VDB
980 _ _ |a I:(DE-Juel1)IAS-8-20210421
980 _ _ |a I:(DE-Juel1)IAS-6-20130828
980 _ _ |a UNRESTRICTED
980 1 _ |a FullTexts


LibraryCollectionCLSMajorCLSMinorLanguageAuthor
Marc 21