001047276 001__ 1047276
001047276 005__ 20251023202111.0
001047276 037__ $$aFZJ-2025-04197
001047276 041__ $$aEnglish
001047276 1001_ $$0P:(DE-Juel1)208793$$aDraganov, Andrew Alexander$$b0$$ufzj
001047276 1112_ $$a42 nd International Conference on Machine Learning$$cVancouver, Canada.$$d2025-07-13 - 2025-07-19$$wCanada
001047276 245__ $$aOn the Importance of Embedding Norms in Self-Supervised Learning
001047276 250__ $$aPMLR 267, 2025
001047276 260__ $$c2025
001047276 3367_ $$2DRIVER$$aconferenceObject
001047276 3367_ $$0PUB:(DE-HGF)26$$2PUB:(DE-HGF)$$aProceedings$$bproc$$mproc$$s1761206663_16747
001047276 3367_ $$2DataCite$$aOutput Types/Book
001047276 3367_ $$2ORCID$$aBOOK
001047276 3367_ $$03$$2EndNote$$aConference Proceedings
001047276 3367_ $$2BibTeX$$aPROCEEDINGS
001047276 4900_ $$v267
001047276 500__ $$ahttps://icml.cc/media/PosterPDFs/ICML%202025/45610.png?t=1751969010.7689042
001047276 520__ $$aAbstractSelf-supervised learning (SSL) allows training data representations without a supervised signal and has become an important paradigm in machine learning. Most SSL methods employ the cosine similarity between embedding vectors and hence effectively embed data on a hypersphere. While this seemingly implies that embedding norms cannot play any role in SSL, a few recent works have suggested that embedding norms have properties related to network convergence and confidence. In this paper, we resolve this apparent contradiction and systematically establish the embedding norm’s role in SSL training. Using theoretical analysis, simulations, and experiments, we show that embedding norms (i) govern SSL convergence rates and (ii) encode network confidence, with smaller norms corresponding to unexpected samples. Additionally, we show that manipulating embedding norms can have large effects on convergence speed. Our findings demonstrate that SSL embedding norms are integral to understanding and optimizing network behavior.
001047276 536__ $$0G:(DE-HGF)POF4-5112$$a5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511)$$cPOF4-511$$fPOF IV$$x0
001047276 7001_ $$0P:(DE-HGF)0$$aSharvaree, Vadgama$$b1
001047276 7001_ $$0P:(DE-HGF)0$$aSebastian, Damrich$$b2
001047276 7001_ $$0P:(DE-HGF)0$$aBöhm, Jan Niklas$$b3
001047276 7001_ $$0P:(DE-HGF)0$$aMaes, Lucas$$b4
001047276 7001_ $$0P:(DE-HGF)0$$aKobak, Dmitry$$b5
001047276 7001_ $$0P:(DE-HGF)0$$aBekkers, Erik$$b6
001047276 909CO $$ooai:juser.fz-juelich.de:1047276$$pVDB
001047276 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)208793$$aForschungszentrum Jülich$$b0$$kFZJ
001047276 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a AMLab, University of Amsterdam, The Netherlands$$b1
001047276 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a 4Hertie Institute for AI in Brain Health, University of T¨ubingen, Germany$$b2
001047276 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a 4Hertie Institute for AI in Brain Health, University of T¨ubingen, Germany$$b3
001047276 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a 5Mila, Quebec AI Institute, Canada.$$b4
001047276 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a 4Hertie Institute for AI in Brain Health, University of T¨ubingen, Germany$$b5
001047276 9101_ $$0I:(DE-HGF)0$$6P:(DE-HGF)0$$a AMLab, University of Amsterdam, The Netherlands$$b6
001047276 9131_ $$0G:(DE-HGF)POF4-511$$1G:(DE-HGF)POF4-510$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5112$$aDE-HGF$$bKey Technologies$$lEngineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action$$vEnabling Computational- & Data-Intensive Science and Engineering$$x0
001047276 9141_ $$y2025
001047276 920__ $$lyes
001047276 9201_ $$0I:(DE-Juel1)IAS-8-20210421$$kIAS-8$$lDatenanalyse und Maschinenlernen$$x0
001047276 980__ $$aproc
001047276 980__ $$aVDB
001047276 980__ $$aI:(DE-Juel1)IAS-8-20210421
001047276 980__ $$aUNRESTRICTED