001034463 001__ 1034463
001034463 005__ 20250203103235.0
001034463 0247_ $$2doi$$a10.48550/ARXIV.2305.13562
001034463 037__ $$aFZJ-2024-07229
001034463 1001_ $$0P:(DE-Juel1)188273$$aNeftci, Emre$$b0$$ufzj
001034463 1112_ $$aThe Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24)$$cVancouver$$d2024-02-20 - 2024-02-27$$wCanada
001034463 245__ $$aUnderstanding and Improving Optimization in Predictive Coding Networks
001034463 260__ $$barXiv$$c2023
001034463 300__ $$a10812-10820
001034463 3367_ $$2ORCID$$aCONFERENCE_PAPER
001034463 3367_ $$033$$2EndNote$$aConference Paper
001034463 3367_ $$2BibTeX$$aINPROCEEDINGS
001034463 3367_ $$2DRIVER$$aconferenceObject
001034463 3367_ $$2DataCite$$aOutput Types/Conference Paper
001034463 3367_ $$0PUB:(DE-HGF)8$$2PUB:(DE-HGF)$$aContribution to a conference proceedings$$bcontrib$$mcontrib$$s1734423736_31719
001034463 520__ $$aBackpropagation (BP), the standard learning algorithm for artificial neural networks, is often considered biologically implausible. In contrast, the standard learning algorithm for predictive coding (PC) models in neuroscience, known as the inference learning algorithm (IL), is a promising, bio-plausible alternative. However, several challenges and questions hinder IL's application to real-world problems. For example, IL is computationally demanding, and without memory-intensive optimizers like Adam, IL may converge to poor local minima. Moreover, although IL can reduce loss more quickly than BP, the reasons for these speedups or their robustness remains unclear. In this paper, we tackle these challenges by 1) altering the standard implementation of PC circuits to substantially reduce computation, 2) developing a novel optimizer that improves the convergence of IL without increasing memory usage, and 3) establishing theoretical results that help elucidate the conditions under which IL is sensitive to second and higher-order information.
001034463 536__ $$0G:(DE-HGF)POF4-5234$$a5234 - Emerging NC Architectures (POF4-523)$$cPOF4-523$$fPOF IV$$x0
001034463 588__ $$aDataset connected to DataCite
001034463 650_7 $$2Other$$aNeural and Evolutionary Computing (cs.NE)
001034463 650_7 $$2Other$$aNeurons and Cognition (q-bio.NC)
001034463 650_7 $$2Other$$aFOS: Computer and information sciences
001034463 650_7 $$2Other$$aFOS: Biological sciences
001034463 7001_ $$0P:(DE-HGF)0$$aJeffrey, Krichmar$$b1
001034463 7001_ $$0P:(DE-HGF)0$$aAlonso, Nicholas$$b2
001034463 773__ $$a10.48550/ARXIV.2305.13562
001034463 8564_ $$uhttps://juser.fz-juelich.de/record/1034463/files/28954-Article%20Text-33008-1-2-20240324.pdf$$yRestricted
001034463 909CO $$ooai:juser.fz-juelich.de:1034463$$pVDB
001034463 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)188273$$aForschungszentrum Jülich$$b0$$kFZJ
001034463 9131_ $$0G:(DE-HGF)POF4-523$$1G:(DE-HGF)POF4-520$$2G:(DE-HGF)POF4-500$$3G:(DE-HGF)POF4$$4G:(DE-HGF)POF$$9G:(DE-HGF)POF4-5234$$aDE-HGF$$bKey Technologies$$lNatural, Artificial and Cognitive Information Processing$$vNeuromorphic Computing and Network Dynamics$$x0
001034463 9141_ $$y2024
001034463 920__ $$lno
001034463 9201_ $$0I:(DE-Juel1)PGI-15-20210701$$kPGI-15$$lNeuromorphic Software Eco System$$x0
001034463 980__ $$acontrib
001034463 980__ $$aVDB
001034463 980__ $$aI:(DE-Juel1)PGI-15-20210701
001034463 980__ $$aUNRESTRICTED