001038128 001__ 1038128
001038128 005__ 20250130220644.0
001038128 0247_ $$2datacite_doi$$a10.34734/FZJ-2025-01175
001038128 037__ $$aFZJ-2025-01175
001038128 1001_ $$0P:(DE-Juel1)192147$$aLohoff, Jamie$$b0$$ufzj
001038128 1112_ $$aNeuro-Inspired Computational Elements$$cHeidelberg$$d2025-03-25 - 2025-03-28$$gNICE$$wGermany
001038128 245__ $$aA Truly Sparse and General Implementation ofGradient-Based Synaptic Plasticity
001038128 260__ $$c2025
001038128 300__ $$an/a
001038128 3367_ $$2ORCID$$aCONFERENCE_PAPER
001038128 3367_ $$033$$2EndNote$$aConference Paper
001038128 3367_ $$2BibTeX$$aINPROCEEDINGS
001038128 3367_ $$2DRIVER$$aconferenceObject
001038128 3367_ $$2DataCite$$aOutput Types/Conference Paper
001038128 3367_ $$0PUB:(DE-HGF)8$$2PUB:(DE-HGF)$$aContribution to a conference proceedings$$bcontrib$$mcontrib$$s1738249323_31383
001038128 500__ $$aAccepted as an oral presentation.
001038128 520__ $$aOnline synaptic plasticity rules derived from gradi-ent descent achieve high accuracy on a wide range of practicaltasks. However, their software implementation often requirestediously hand-derived gradients or using gradient backprop-agation which sacrifices the online capability of the rules. Inthis work, we present a custom automatic differentiation (AD)pipeline for sparse and online implementation of gradient-based synaptic plasticity rules that generalizes to arbitraryneuron models. Our work combines the programming easeof backpropagation-type methods for forward AD while beingmemory-efficient. To achieve this, we exploit the advantageouscompute and memory scaling of online synaptic plasticity byproviding an inherently sparse implementation of AD whereexpensive tensor contractions are replaced with simple element-wise multiplications if the tensors are diagonal. Gradient-basedsynaptic plasticity rules such as eligibility propagation (e-prop)have exactly this property and thus profit immensely from thisfeature. We demonstrate the alignment of our gradients withrespect to gradient backpropagation on an synthetic task wheree-prop gradients are exact, as well as audio speech classificationbenchmarks. We demonstrate how memory utilization scales withnetwork size without dependence on the sequence length, asexpected from forward AD methods.
001038128 536__ $$0G:(DE-HGF)POF4-5234$$a5234 - Emerging NC Architectures (POF4-523)$$cPOF4-523$$fPOF IV$$x0
001038128 536__ $$0G:(BMBF)16ME0400$$aBMBF 16ME0400 - Verbundprojekt: Neuro-inspirierte Technologien der künstlichen Intelligenz für die Elektronik der Zukunft - NEUROTEC II - (16ME0400)$$c16ME0400$$x1
001038128 536__ $$0G:(EU-Grant)953775$$aGREENEDGE - Taming the environmental impact of mobile networks through GREEN EDGE computing platforms (953775)$$c953775$$fH2020-MSCA-ITN-2020$$x2
001038128 7001_ $$0P:(DE-Juel1)204426$$aKaya, Anil$$b1$$ufzj
001038128 7001_ $$0P:(DE-Juel1)206804$$aAssmuth, Florian$$b2$$ufzj
001038128 7001_ $$0P:(DE-Juel1)188273$$aNeftci, Emre$$b3$$ufzj
001038128 8564_ $$uhttps://juser.fz-juelich.de/record/1038128/files/SparseEprop.pdf$$yOpenAccess
001038128 909CO $$ooai:juser.fz-juelich.de:1038128$$popenaire$$popen_access$$pdriver$$pVDB$$pec_fundedresources$$pdnbdelivery
001038128 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)192147$$aForschungszentrum Jülich$$b0$$kFZJ
001038128 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)204426$$aForschungszentrum Jülich$$b1$$kFZJ
001038128 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)206804$$aForschungszentrum Jülich$$b2$$kFZJ
001038128 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)188273$$aForschungszentrum Jülich$$b3$$kFZJ
001038128 9131_ $$0G:(DE-HGF)POF4-523$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5234$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vNeuromorphic Computing and Network Dynamics$$x0
001038128 9141_ $$y2025
001038128 915__ $$0StatID:(DE-HGF)0510$$2StatID$$aOpenAccess
001038128 920__ $$lyes
001038128 9201_ $$0I:(DE-Juel1)PGI-15-20210701$$kPGI-15$$lNeuromorphic Software Eco System$$x0
001038128 980__ $$acontrib
001038128 980__ $$aVDB
001038128 980__ $$aUNRESTRICTED
001038128 980__ $$aI:(DE-Juel1)PGI-15-20210701
001038128 9801_ $$aFullTexts