guest ::
| Home > Publications database > Understanding and Improving Optimization in Predictive Coding Networks > print |
| Home > Publications database > Understanding and Improving Optimization in Predictive Coding Networks > print |
| 001 | 1034463 | ||
| 005 | 20250203103235.0 | ||
| 024 | 7 | _ | |a 10.48550/ARXIV.2305.13562 |2 doi |
| 037 | _ | _ | |a FZJ-2024-07229 |
| 100 | 1 | _ | |a Neftci, Emre |0 P:(DE-Juel1)188273 |b 0 |u fzj |
| 111 | 2 | _ | |a The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) |c Vancouver |d 2024-02-20 - 2024-02-27 |w Canada |
| 245 | _ | _ | |a Understanding and Improving Optimization in Predictive Coding Networks |
| 260 | _ | _ | |c 2023 |b arXiv |
| 300 | _ | _ | |a 10812-10820 |
| 336 | 7 | _ | |a CONFERENCE_PAPER |2 ORCID |
| 336 | 7 | _ | |a Conference Paper |0 33 |2 EndNote |
| 336 | 7 | _ | |a INPROCEEDINGS |2 BibTeX |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a Output Types/Conference Paper |2 DataCite |
| 336 | 7 | _ | |a Contribution to a conference proceedings |b contrib |m contrib |0 PUB:(DE-HGF)8 |s 1734423736_31719 |2 PUB:(DE-HGF) |
| 520 | _ | _ | |a Backpropagation (BP), the standard learning algorithm for artificial neural networks, is often considered biologically implausible. In contrast, the standard learning algorithm for predictive coding (PC) models in neuroscience, known as the inference learning algorithm (IL), is a promising, bio-plausible alternative. However, several challenges and questions hinder IL's application to real-world problems. For example, IL is computationally demanding, and without memory-intensive optimizers like Adam, IL may converge to poor local minima. Moreover, although IL can reduce loss more quickly than BP, the reasons for these speedups or their robustness remains unclear. In this paper, we tackle these challenges by 1) altering the standard implementation of PC circuits to substantially reduce computation, 2) developing a novel optimizer that improves the convergence of IL without increasing memory usage, and 3) establishing theoretical results that help elucidate the conditions under which IL is sensitive to second and higher-order information. |
| 536 | _ | _ | |a 5234 - Emerging NC Architectures (POF4-523) |0 G:(DE-HGF)POF4-5234 |c POF4-523 |f POF IV |x 0 |
| 588 | _ | _ | |a Dataset connected to DataCite |
| 650 | _ | 7 | |a Neural and Evolutionary Computing (cs.NE) |2 Other |
| 650 | _ | 7 | |a Neurons and Cognition (q-bio.NC) |2 Other |
| 650 | _ | 7 | |a FOS: Computer and information sciences |2 Other |
| 650 | _ | 7 | |a FOS: Biological sciences |2 Other |
| 700 | 1 | _ | |a Jeffrey, Krichmar |0 P:(DE-HGF)0 |b 1 |
| 700 | 1 | _ | |a Alonso, Nicholas |0 P:(DE-HGF)0 |b 2 |
| 773 | _ | _ | |a 10.48550/ARXIV.2305.13562 |
| 856 | 4 | _ | |u https://juser.fz-juelich.de/record/1034463/files/28954-Article%20Text-33008-1-2-20240324.pdf |y Restricted |
| 909 | C | O | |o oai:juser.fz-juelich.de:1034463 |p VDB |
| 910 | 1 | _ | |a Forschungszentrum Jülich |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)188273 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Natural, Artificial and Cognitive Information Processing |1 G:(DE-HGF)POF4-520 |0 G:(DE-HGF)POF4-523 |3 G:(DE-HGF)POF4 |2 G:(DE-HGF)POF4-500 |4 G:(DE-HGF)POF |v Neuromorphic Computing and Network Dynamics |9 G:(DE-HGF)POF4-5234 |x 0 |
| 914 | 1 | _ | |y 2024 |
| 920 | _ | _ | |l no |
| 920 | 1 | _ | |0 I:(DE-Juel1)PGI-15-20210701 |k PGI-15 |l Neuromorphic Software Eco System |x 0 |
| 980 | _ | _ | |a contrib |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a I:(DE-Juel1)PGI-15-20210701 |
| 980 | _ | _ | |a UNRESTRICTED |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|