Gast ::
| Hauptseite > Publikationsdatenbank > A Truly Sparse and General Implementation ofGradient-Based Synaptic Plasticity > print |
| Hauptseite > Publikationsdatenbank > A Truly Sparse and General Implementation ofGradient-Based Synaptic Plasticity > print |
| 001 | 1038128 | ||
| 005 | 20250130220644.0 | ||
| 024 | 7 | _ | |a 10.34734/FZJ-2025-01175 |2 datacite_doi |
| 037 | _ | _ | |a FZJ-2025-01175 |
| 100 | 1 | _ | |a Lohoff, Jamie |0 P:(DE-Juel1)192147 |b 0 |u fzj |
| 111 | 2 | _ | |a Neuro-Inspired Computational Elements |g NICE |c Heidelberg |d 2025-03-25 - 2025-03-28 |w Germany |
| 245 | _ | _ | |a A Truly Sparse and General Implementation ofGradient-Based Synaptic Plasticity |
| 260 | _ | _ | |c 2025 |
| 300 | _ | _ | |a n/a |
| 336 | 7 | _ | |a CONFERENCE_PAPER |2 ORCID |
| 336 | 7 | _ | |a Conference Paper |0 33 |2 EndNote |
| 336 | 7 | _ | |a INPROCEEDINGS |2 BibTeX |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a Output Types/Conference Paper |2 DataCite |
| 336 | 7 | _ | |a Contribution to a conference proceedings |b contrib |m contrib |0 PUB:(DE-HGF)8 |s 1738249323_31383 |2 PUB:(DE-HGF) |
| 500 | _ | _ | |a Accepted as an oral presentation. |
| 520 | _ | _ | |a Online synaptic plasticity rules derived from gradi-ent descent achieve high accuracy on a wide range of practicaltasks. However, their software implementation often requirestediously hand-derived gradients or using gradient backprop-agation which sacrifices the online capability of the rules. Inthis work, we present a custom automatic differentiation (AD)pipeline for sparse and online implementation of gradient-based synaptic plasticity rules that generalizes to arbitraryneuron models. Our work combines the programming easeof backpropagation-type methods for forward AD while beingmemory-efficient. To achieve this, we exploit the advantageouscompute and memory scaling of online synaptic plasticity byproviding an inherently sparse implementation of AD whereexpensive tensor contractions are replaced with simple element-wise multiplications if the tensors are diagonal. Gradient-basedsynaptic plasticity rules such as eligibility propagation (e-prop)have exactly this property and thus profit immensely from thisfeature. We demonstrate the alignment of our gradients withrespect to gradient backpropagation on an synthetic task wheree-prop gradients are exact, as well as audio speech classificationbenchmarks. We demonstrate how memory utilization scales withnetwork size without dependence on the sequence length, asexpected from forward AD methods. |
| 536 | _ | _ | |a 5234 - Emerging NC Architectures (POF4-523) |0 G:(DE-HGF)POF4-5234 |c POF4-523 |f POF IV |x 0 |
| 536 | _ | _ | |a BMBF 16ME0400 - Verbundprojekt: Neuro-inspirierte Technologien der künstlichen Intelligenz für die Elektronik der Zukunft - NEUROTEC II - (16ME0400) |0 G:(BMBF)16ME0400 |c 16ME0400 |x 1 |
| 536 | _ | _ | |a GREENEDGE - Taming the environmental impact of mobile networks through GREEN EDGE computing platforms (953775) |0 G:(EU-Grant)953775 |c 953775 |f H2020-MSCA-ITN-2020 |x 2 |
| 700 | 1 | _ | |a Kaya, Anil |0 P:(DE-Juel1)204426 |b 1 |u fzj |
| 700 | 1 | _ | |a Assmuth, Florian |0 P:(DE-Juel1)206804 |b 2 |u fzj |
| 700 | 1 | _ | |a Neftci, Emre |0 P:(DE-Juel1)188273 |b 3 |u fzj |
| 856 | 4 | _ | |u https://juser.fz-juelich.de/record/1038128/files/SparseEprop.pdf |y OpenAccess |
| 909 | C | O | |o oai:juser.fz-juelich.de:1038128 |p openaire |p open_access |p driver |p VDB |p ec_fundedresources |p dnbdelivery |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)192147 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 1 |6 P:(DE-Juel1)204426 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 2 |6 P:(DE-Juel1)206804 |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 3 |6 P:(DE-Juel1)188273 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Natural, Artificial and Cognitive Information Processing |1 G:(DE-HGF)POF4-520 |0 G:(DE-HGF)POF4-523 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Neuromorphic Computing and Network Dynamics |9 G:(DE-HGF)POF4-5234 |x 0 |
| 914 | 1 | _ | |y 2025 |
| 915 | _ | _ | |a OpenAccess |0 StatID:(DE-HGF)0510 |2 StatID |
| 920 | _ | _ | |l yes |
| 920 | 1 | _ | |0 I:(DE-Juel1)PGI-15-20210701 |k PGI-15 |l Neuromorphic Software Eco System |x 0 |
| 980 | _ | _ | |a contrib |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a UNRESTRICTED |
| 980 | _ | _ | |a I:(DE-Juel1)PGI-15-20210701 |
| 980 | 1 | _ | |a FullTexts |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|