000255885 001__ 255885
000255885 005__ 20240313094902.0
000255885 037__ $$aFZJ-2015-05991
000255885 041__ $$aEnglish
000255885 1001_ $$0P:(DE-Juel1)151356$$aJordan, Jakob$$b0$$eCorresponding author
000255885 1112_ $$aCNS$$cPrague$$d2015-07-18 - 2015-07-23$$wChech Republic
000255885 245__ $$aDeterministic neural networks as sources of uncorrelated noise for probabilistic computations
000255885 260__ $$c2015
000255885 3367_ $$0PUB:(DE-HGF)1$$2PUB:(DE-HGF)$$aAbstract$$babstract$$mabstract$$s1444398903_22597
000255885 3367_ $$033$$2EndNote$$aConference Paper
000255885 3367_ $$2DataCite$$aOutput Types/Conference Abstract
000255885 3367_ $$2ORCID$$aOTHER
000255885 3367_ $$2DRIVER$$aconferenceObject
000255885 3367_ $$2BibTeX$$aINPROCEEDINGS
000255885 520__ $$aNeural-network models of brain function often rely on the presence ofnoise [1-5]. To date, the interplay of microscopic noise sourcesand network function is only poorly understood. In computersimulations and in neuromorphic hardware [6-8], the number of noisesources (random-number generators) is limited. In consequence, neuronsin large functional network models have to share noise sources and aretherefore correlated. In general, it is unclear how shared-noisecorrelations affect the performance of functional networkmodels. Further, there is so far no solution to the problem of how alimited number of noise sources can supply a large number offunctional units with uncorrelated noise.Here, we investigate the performance of neural Boltzmann machines[2-4]. We show that correlations in the background activity aredetrimental to the sampling performance and that the deviations fromthe target distribution scale inversely with the number of noisesources. Further, we show that this problem can be overcome byreplacing the finite ensemble of independent noise sources by arecurrent neural network with the same number of units. As shownrecently, inhibitory feedback, abundant in biological neural networks,serves as a powerful decorrelation mechanism [9,10]: Shared-noisecorrelations are actively suppressed by the network dynamics. Byexploiting this effect, the network performance is significantlyimproved. Hence, recurrent neural networks can serve as naturalfinite-size noise sources for functional neural networks, both inbiological and in synthetic neuromorphic substrates. Finally weinvestigate the impact of sampling network parameters on its abilityto faithfully represent a given well-defined distribution. We showthat sampling networks with sufficiently strong negative feedback canintrinsically suppress correlations in the background activity, andthereby improve their performance substantially.Acknowledgments: Partially supported by the Helmholtz Associationportfolio theme SMHB, the Jülich Aachen Research Alliance (JARA), EUGrant 269921 (BrainScaleS), The Austrian Science Fund FWF #I753-N23(PNEUMA), The Manfred Stärk Foundation, and EU Grant 604102 (HumanBrain Project, HBP).[1] Rolls ET, Deco G: The noisy brain. Oxford University Press, 2010[2] Hinton GE, Sejnowski TJ, Ackley DH: Boltzmann machines: constraintsatisfaction networks that learn. Technical report, Carnegie-MellonUniversity, 1984[3] Buesing L, Bill J, Nessler B, Maass W: Neural Dynamics asSampling: A Model for Stochastic Computation in Recurrent Networks ofSpiking Neurons. PloS CB, 2011, 7, e1002211.[4] Petrovici MA, Bill J, Bytschok I, Schemmel J, Meier K: Stochasticinference with deterministic spiking neurons. arXiv, 2013, 1311.3211v1[q-bio.NC][5] Probst D, Petrovici MA, Bytschok I, Bill J, Pecevski D, SchemmelJ, Meier K: Probabilistic inference in discrete spaces can beimplemented into networks of LIF neurons. Front. Comput. Neurosci.,2015, 9:13.[6] Schemmel J, Bruederle D, Gruebl A, Hock M, Meier K, Millner S: AWafer-Scale Neuromorphic Hardware System for Large-Scale NeuralModeling. Proceedings of the 2010 International Symposium on Circuitsand Systems (ISCAS), IEEE Press, 2010, 1947-1950[7] Bruederle D, Petrovici M, Vogginger B, Ehrlich M, Pfeil T, MillnerS, Gruebl A, Wendt K, Mueller E, Schwartz MO et al.: A comprehensiveworkflow for general-purpose neural modeling with highly configurableneuromorphic hardware systems. Biological Cybernetics, 2011, 104,263-296[8] Petrovici MA, Vogginger B, Mueller P, Breitwieser O, Lundqvist M,Muller L, Ehrlich M, Destexhe A, Lansner A, Schueffny R et al.:Characterization and Compensation of Network-Level Anomalies inMixed-Signal Neuromorphic Modeling Platforms. PLoS ONE, 2014, 9(10):e108590.[9] Renart A, De La Rocha J, Bartho P, Hollender L, Parga N, Reyes A,Harris KD: The asynchronous State in Cortical Circuits. Science, 2010,327: 587-590[10] Tetzlaff T, Helias M, Einevoll G, Diesmann M: Decorrelation ofneural-network activity by inhibitory feedback. PloS CB, 2012, 8,e1002596
000255885 536__ $$0G:(DE-HGF)POF3-574$$a574 - Theory, modelling and simulation (POF3-574)$$cPOF3-574$$fPOF III$$x0
000255885 536__ $$0G:(DE-HGF)POF2-899$$a899 - ohne Topic (POF2-899)$$cPOF2-899$$fPOF I$$x1
000255885 536__ $$0G:(DE-Juel1)HGF-SMHB-2013-2017$$aSMHB - Supercomputing and Modelling for the Human Brain (HGF-SMHB-2013-2017)$$cHGF-SMHB-2013-2017$$fSMHB$$x2
000255885 536__ $$0G:(EU-Grant)604102$$aHBP - The Human Brain Project (604102)$$c604102$$fFP7-ICT-2013-FET-F$$x3
000255885 536__ $$0G:(EU-Grant)269921$$aBRAINSCALES - Brain-inspired multiscale computation in neuromorphic hybrid systems (269921)$$c269921$$fFP7-ICT-2009-6$$x4
000255885 7001_ $$0P:(DE-HGF)0$$aPetrovici, Mihai$$b1
000255885 7001_ $$0P:(DE-HGF)0$$aPfeil, Thomas$$b2
000255885 7001_ $$0P:(DE-HGF)0$$aBreitwieser, Oliver$$b3
000255885 7001_ $$0P:(DE-HGF)0$$aBytschok, Ilja$$b4
000255885 7001_ $$0P:(DE-HGF)0$$aBill, Johannes$$b5
000255885 7001_ $$0P:(DE-HGF)0$$aGruebl, Andreas$$b6
000255885 7001_ $$0P:(DE-HGF)0$$aSchemmel, Johannes$$b7
000255885 7001_ $$0P:(DE-HGF)0$$aMeier, Karlheinz$$b8
000255885 7001_ $$0P:(DE-Juel1)144174$$aDiesmann, Markus$$b9
000255885 7001_ $$0P:(DE-Juel1)145211$$aTetzlaff, Tom$$b10
000255885 909CO $$ooai:juser.fz-juelich.de:255885$$popenaire$$pec_fundedresources$$pVDB
000255885 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)151356$$aForschungszentrum Jülich GmbH$$b0$$kFZJ
000255885 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)144174$$aForschungszentrum Jülich GmbH$$b9$$kFZJ
000255885 9101_ $$0I:(DE-588b)5008462-8$$6P:(DE-Juel1)145211$$aForschungszentrum Jülich GmbH$$b10$$kFZJ
000255885 9132_ $$0G:(DE-HGF)POF3-574$$1G:(DE-HGF)POF3-570$$2G:(DE-HGF)POF3-500$$aDE-HGF$$bKey Technologies$$lDecoding the Human Brain$$vTheory, modelling and simulation$$x0
000255885 9131_ $$0G:(DE-HGF)POF3-574$$1G:(DE-HGF)POF3-570$$2G:(DE-HGF)POF3-500$$3G:(DE-HGF)POF3$$4G:(DE-HGF)POF$$aDE-HGF$$bKey Technologies$$lDecoding the Human Brain$$vTheory, modelling and simulation$$x0
000255885 9131_ $$0G:(DE-HGF)POF2-899$$1G:(DE-HGF)POF2-890$$2G:(DE-HGF)POF2-800$$3G:(DE-HGF)POF2$$4G:(DE-HGF)POF$$aDE-HGF$$bProgrammungebundene Forschung$$lohne Programm$$vohne Topic$$x1
000255885 9141_ $$y2015
000255885 9201_ $$0I:(DE-Juel1)INM-6-20090406$$kINM-6$$lComputational and Systems Neuroscience$$x0
000255885 9201_ $$0I:(DE-Juel1)IAS-6-20130828$$kIAS-6$$lTheoretical Neuroscience$$x1
000255885 980__ $$aabstract
000255885 980__ $$aVDB
000255885 980__ $$aI:(DE-Juel1)INM-6-20090406
000255885 980__ $$aI:(DE-Juel1)IAS-6-20130828
000255885 980__ $$aUNRESTRICTED
000255885 981__ $$aI:(DE-Juel1)IAS-6-20130828
000255885 981__ $$aI:(DE-Juel1)IAS-6-20130828