guest ::
| Home > Publications database > On the Importance of Embedding Norms in Self-Supervised Learning > print |
| Home > Publications database > On the Importance of Embedding Norms in Self-Supervised Learning > print |
| 001 | 1047276 | ||
| 005 | 20251023202111.0 | ||
| 037 | _ | _ | |a FZJ-2025-04197 |
| 041 | _ | _ | |a English |
| 100 | 1 | _ | |a Draganov, Andrew Alexander |0 P:(DE-Juel1)208793 |b 0 |u fzj |
| 111 | 2 | _ | |a 42 nd International Conference on Machine Learning |c Vancouver, Canada. |d 2025-07-13 - 2025-07-19 |w Canada |
| 245 | _ | _ | |a On the Importance of Embedding Norms in Self-Supervised Learning |
| 250 | _ | _ | |a PMLR 267, 2025 |
| 260 | _ | _ | |c 2025 |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a Proceedings |b proc |m proc |0 PUB:(DE-HGF)26 |s 1761206663_16747 |2 PUB:(DE-HGF) |
| 336 | 7 | _ | |a Output Types/Book |2 DataCite |
| 336 | 7 | _ | |a BOOK |2 ORCID |
| 336 | 7 | _ | |a Conference Proceedings |0 3 |2 EndNote |
| 336 | 7 | _ | |a PROCEEDINGS |2 BibTeX |
| 490 | 0 | _ | |v 267 |
| 500 | _ | _ | |a https://icml.cc/media/PosterPDFs/ICML%202025/45610.png?t=1751969010.7689042 |
| 520 | _ | _ | |a AbstractSelf-supervised learning (SSL) allows training data representations without a supervised signal and has become an important paradigm in machine learning. Most SSL methods employ the cosine similarity between embedding vectors and hence effectively embed data on a hypersphere. While this seemingly implies that embedding norms cannot play any role in SSL, a few recent works have suggested that embedding norms have properties related to network convergence and confidence. In this paper, we resolve this apparent contradiction and systematically establish the embedding norm’s role in SSL training. Using theoretical analysis, simulations, and experiments, we show that embedding norms (i) govern SSL convergence rates and (ii) encode network confidence, with smaller norms corresponding to unexpected samples. Additionally, we show that manipulating embedding norms can have large effects on convergence speed. Our findings demonstrate that SSL embedding norms are integral to understanding and optimizing network behavior. |
| 536 | _ | _ | |a 5112 - Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups (POF4-511) |0 G:(DE-HGF)POF4-5112 |c POF4-511 |f POF IV |x 0 |
| 700 | 1 | _ | |a Sharvaree, Vadgama |0 P:(DE-HGF)0 |b 1 |
| 700 | 1 | _ | |a Sebastian, Damrich |0 P:(DE-HGF)0 |b 2 |
| 700 | 1 | _ | |a Böhm, Jan Niklas |0 P:(DE-HGF)0 |b 3 |
| 700 | 1 | _ | |a Maes, Lucas |0 P:(DE-HGF)0 |b 4 |
| 700 | 1 | _ | |a Kobak, Dmitry |0 P:(DE-HGF)0 |b 5 |
| 700 | 1 | _ | |a Bekkers, Erik |0 P:(DE-HGF)0 |b 6 |
| 909 | C | O | |o oai:juser.fz-juelich.de:1047276 |p VDB |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)208793 |
| 910 | 1 | _ | |a AMLab, University of Amsterdam, The Netherlands |0 I:(DE-HGF)0 |b 1 |6 P:(DE-HGF)0 |
| 910 | 1 | _ | |a 4Hertie Institute for AI in Brain Health, University of T¨ubingen, Germany |0 I:(DE-HGF)0 |b 2 |6 P:(DE-HGF)0 |
| 910 | 1 | _ | |a 4Hertie Institute for AI in Brain Health, University of T¨ubingen, Germany |0 I:(DE-HGF)0 |b 3 |6 P:(DE-HGF)0 |
| 910 | 1 | _ | |a 5Mila, Quebec AI Institute, Canada. |0 I:(DE-HGF)0 |b 4 |6 P:(DE-HGF)0 |
| 910 | 1 | _ | |a 4Hertie Institute for AI in Brain Health, University of T¨ubingen, Germany |0 I:(DE-HGF)0 |b 5 |6 P:(DE-HGF)0 |
| 910 | 1 | _ | |a AMLab, University of Amsterdam, The Netherlands |0 I:(DE-HGF)0 |b 6 |6 P:(DE-HGF)0 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action |1 G:(DE-HGF)POF4-510 |0 G:(DE-HGF)POF4-511 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Enabling Computational- & Data-Intensive Science and Engineering |9 G:(DE-HGF)POF4-5112 |x 0 |
| 914 | 1 | _ | |y 2025 |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)IAS-8-20210421 |k IAS-8 |l Datenanalyse und Maschinenlernen |x 0 |
| 980 | _ | _ | |a proc |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a I:(DE-Juel1)IAS-8-20210421 |
| 980 | _ | _ | |a UNRESTRICTED |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|