Gast ::
| Hauptseite > Publikationsdatenbank > Deterministic neural networks as sources of uncorrelated noise for probabilistic computations > print |
| Hauptseite > Publikationsdatenbank > Deterministic neural networks as sources of uncorrelated noise for probabilistic computations > print |
| 001 | 255885 | ||
| 005 | 20240313094902.0 | ||
| 037 | _ | _ | |a FZJ-2015-05991 |
| 041 | _ | _ | |a English |
| 100 | 1 | _ | |a Jordan, Jakob |0 P:(DE-Juel1)151356 |b 0 |e Corresponding author |
| 111 | 2 | _ | |a CNS |c Prague |d 2015-07-18 - 2015-07-23 |w Chech Republic |
| 245 | _ | _ | |a Deterministic neural networks as sources of uncorrelated noise for probabilistic computations |
| 260 | _ | _ | |c 2015 |
| 336 | 7 | _ | |a Abstract |b abstract |m abstract |0 PUB:(DE-HGF)1 |s 1444398903_22597 |2 PUB:(DE-HGF) |
| 336 | 7 | _ | |a Conference Paper |0 33 |2 EndNote |
| 336 | 7 | _ | |a Output Types/Conference Abstract |2 DataCite |
| 336 | 7 | _ | |a OTHER |2 ORCID |
| 336 | 7 | _ | |a conferenceObject |2 DRIVER |
| 336 | 7 | _ | |a INPROCEEDINGS |2 BibTeX |
| 520 | _ | _ | |a Neural-network models of brain function often rely on the presence ofnoise [1-5]. To date, the interplay of microscopic noise sourcesand network function is only poorly understood. In computersimulations and in neuromorphic hardware [6-8], the number of noisesources (random-number generators) is limited. In consequence, neuronsin large functional network models have to share noise sources and aretherefore correlated. In general, it is unclear how shared-noisecorrelations affect the performance of functional networkmodels. Further, there is so far no solution to the problem of how alimited number of noise sources can supply a large number offunctional units with uncorrelated noise.Here, we investigate the performance of neural Boltzmann machines[2-4]. We show that correlations in the background activity aredetrimental to the sampling performance and that the deviations fromthe target distribution scale inversely with the number of noisesources. Further, we show that this problem can be overcome byreplacing the finite ensemble of independent noise sources by arecurrent neural network with the same number of units. As shownrecently, inhibitory feedback, abundant in biological neural networks,serves as a powerful decorrelation mechanism [9,10]: Shared-noisecorrelations are actively suppressed by the network dynamics. Byexploiting this effect, the network performance is significantlyimproved. Hence, recurrent neural networks can serve as naturalfinite-size noise sources for functional neural networks, both inbiological and in synthetic neuromorphic substrates. Finally weinvestigate the impact of sampling network parameters on its abilityto faithfully represent a given well-defined distribution. We showthat sampling networks with sufficiently strong negative feedback canintrinsically suppress correlations in the background activity, andthereby improve their performance substantially.Acknowledgments: Partially supported by the Helmholtz Associationportfolio theme SMHB, the Jülich Aachen Research Alliance (JARA), EUGrant 269921 (BrainScaleS), The Austrian Science Fund FWF #I753-N23(PNEUMA), The Manfred Stärk Foundation, and EU Grant 604102 (HumanBrain Project, HBP).[1] Rolls ET, Deco G: The noisy brain. Oxford University Press, 2010[2] Hinton GE, Sejnowski TJ, Ackley DH: Boltzmann machines: constraintsatisfaction networks that learn. Technical report, Carnegie-MellonUniversity, 1984[3] Buesing L, Bill J, Nessler B, Maass W: Neural Dynamics asSampling: A Model for Stochastic Computation in Recurrent Networks ofSpiking Neurons. PloS CB, 2011, 7, e1002211.[4] Petrovici MA, Bill J, Bytschok I, Schemmel J, Meier K: Stochasticinference with deterministic spiking neurons. arXiv, 2013, 1311.3211v1[q-bio.NC][5] Probst D, Petrovici MA, Bytschok I, Bill J, Pecevski D, SchemmelJ, Meier K: Probabilistic inference in discrete spaces can beimplemented into networks of LIF neurons. Front. Comput. Neurosci.,2015, 9:13.[6] Schemmel J, Bruederle D, Gruebl A, Hock M, Meier K, Millner S: AWafer-Scale Neuromorphic Hardware System for Large-Scale NeuralModeling. Proceedings of the 2010 International Symposium on Circuitsand Systems (ISCAS), IEEE Press, 2010, 1947-1950[7] Bruederle D, Petrovici M, Vogginger B, Ehrlich M, Pfeil T, MillnerS, Gruebl A, Wendt K, Mueller E, Schwartz MO et al.: A comprehensiveworkflow for general-purpose neural modeling with highly configurableneuromorphic hardware systems. Biological Cybernetics, 2011, 104,263-296[8] Petrovici MA, Vogginger B, Mueller P, Breitwieser O, Lundqvist M,Muller L, Ehrlich M, Destexhe A, Lansner A, Schueffny R et al.:Characterization and Compensation of Network-Level Anomalies inMixed-Signal Neuromorphic Modeling Platforms. PLoS ONE, 2014, 9(10):e108590.[9] Renart A, De La Rocha J, Bartho P, Hollender L, Parga N, Reyes A,Harris KD: The asynchronous State in Cortical Circuits. Science, 2010,327: 587-590[10] Tetzlaff T, Helias M, Einevoll G, Diesmann M: Decorrelation ofneural-network activity by inhibitory feedback. PloS CB, 2012, 8,e1002596 |
| 536 | _ | _ | |a 574 - Theory, modelling and simulation (POF3-574) |0 G:(DE-HGF)POF3-574 |c POF3-574 |f POF III |x 0 |
| 536 | _ | _ | |a 899 - ohne Topic (POF2-899) |0 G:(DE-HGF)POF2-899 |c POF2-899 |x 1 |f POF I |
| 536 | _ | _ | |a SMHB - Supercomputing and Modelling for the Human Brain (HGF-SMHB-2013-2017) |0 G:(DE-Juel1)HGF-SMHB-2013-2017 |c HGF-SMHB-2013-2017 |f SMHB |x 2 |
| 536 | _ | _ | |a HBP - The Human Brain Project (604102) |0 G:(EU-Grant)604102 |c 604102 |f FP7-ICT-2013-FET-F |x 3 |
| 536 | _ | _ | |a BRAINSCALES - Brain-inspired multiscale computation in neuromorphic hybrid systems (269921) |0 G:(EU-Grant)269921 |c 269921 |f FP7-ICT-2009-6 |x 4 |
| 700 | 1 | _ | |a Petrovici, Mihai |0 P:(DE-HGF)0 |b 1 |
| 700 | 1 | _ | |a Pfeil, Thomas |0 P:(DE-HGF)0 |b 2 |
| 700 | 1 | _ | |a Breitwieser, Oliver |0 P:(DE-HGF)0 |b 3 |
| 700 | 1 | _ | |a Bytschok, Ilja |0 P:(DE-HGF)0 |b 4 |
| 700 | 1 | _ | |a Bill, Johannes |0 P:(DE-HGF)0 |b 5 |
| 700 | 1 | _ | |a Gruebl, Andreas |0 P:(DE-HGF)0 |b 6 |
| 700 | 1 | _ | |a Schemmel, Johannes |0 P:(DE-HGF)0 |b 7 |
| 700 | 1 | _ | |a Meier, Karlheinz |0 P:(DE-HGF)0 |b 8 |
| 700 | 1 | _ | |a Diesmann, Markus |0 P:(DE-Juel1)144174 |b 9 |
| 700 | 1 | _ | |a Tetzlaff, Tom |0 P:(DE-Juel1)145211 |b 10 |
| 909 | C | O | |o oai:juser.fz-juelich.de:255885 |p VDB |p ec_fundedresources |p openaire |
| 910 | 1 | _ | |a Forschungszentrum Jülich GmbH |0 I:(DE-588b)5008462-8 |k FZJ |b 0 |6 P:(DE-Juel1)151356 |
| 910 | 1 | _ | |a Forschungszentrum Jülich GmbH |0 I:(DE-588b)5008462-8 |k FZJ |b 9 |6 P:(DE-Juel1)144174 |
| 910 | 1 | _ | |a Forschungszentrum Jülich GmbH |0 I:(DE-588b)5008462-8 |k FZJ |b 10 |6 P:(DE-Juel1)145211 |
| 913 | 2 | _ | |a DE-HGF |b Key Technologies |l Decoding the Human Brain |1 G:(DE-HGF)POF3-570 |0 G:(DE-HGF)POF3-574 |2 G:(DE-HGF)POF3-500 |v Theory, modelling and simulation |x 0 |
| 913 | 1 | _ | |a DE-HGF |b Key Technologies |l Decoding the Human Brain |1 G:(DE-HGF)POF3-570 |0 G:(DE-HGF)POF3-574 |2 G:(DE-HGF)POF3-500 |v Theory, modelling and simulation |x 0 |4 G:(DE-HGF)POF |3 G:(DE-HGF)POF3 |
| 913 | 1 | _ | |a DE-HGF |b Programmungebundene Forschung |l ohne Programm |1 G:(DE-HGF)POF2-890 |0 G:(DE-HGF)POF2-899 |2 G:(DE-HGF)POF2-800 |v ohne Topic |x 1 |4 G:(DE-HGF)POF |3 G:(DE-HGF)POF2 |
| 914 | 1 | _ | |y 2015 |
| 920 | 1 | _ | |0 I:(DE-Juel1)INM-6-20090406 |k INM-6 |l Computational and Systems Neuroscience |x 0 |
| 920 | 1 | _ | |0 I:(DE-Juel1)IAS-6-20130828 |k IAS-6 |l Theoretical Neuroscience |x 1 |
| 980 | _ | _ | |a abstract |
| 980 | _ | _ | |a VDB |
| 980 | _ | _ | |a I:(DE-Juel1)INM-6-20090406 |
| 980 | _ | _ | |a I:(DE-Juel1)IAS-6-20130828 |
| 980 | _ | _ | |a UNRESTRICTED |
| 981 | _ | _ | |a I:(DE-Juel1)IAS-6-20130828 |
| 981 | _ | _ | |a I:(DE-Juel1)IAS-6-20130828 |
| Library | Collection | CLSMajor | CLSMinor | Language | Author |
|---|