Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning

G. M. Hoerzer, R. Legenstein, and W. Maass

Abstract:

This article addresses the question how generic microcircuits of neurons in different parts of the cortex can attain and maintain different computational specializations. We show that if stochastic variations in the dynamics of local microcircuits are correlated with signals related to functional improvements of the brain (e.g. in the control of behavior), the computational operation of these microcircuits can become optimized for specific tasks such as the generation of specific periodic signals, and task-dependent routing of information. Furthermore, we show that working memory can emerge autonomously through reward-modulated Hebbian learning, if needed for specific tasks. Altogether our results suggest that reward-modulated synaptic plasticity can not only optimize network parameters for specific computational tasks, but can also initiate a functional rewiring that re-programs microcircuits, thereby generating diverse computational functions in different generic cortical microcircuits. On a more general level this article provides a new perspective for a standard model for computations in generic cortical microcircuits (liquid computing model). It shows that the arguably most problematic assumption of this model, the postulate of a teacher that trains neural readouts through supervised learning, can be eliminated. We show that generic networks of neurons can learn numerous biologically relevant computations through trial and error.

Reference: G. M. Hoerzer, R. Legenstein, and W. Maass. Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cerebral Cortex, 2012. doi:10.1093/cercor/bhs348, incl. suppl.