Spiking neurons can learn to solve information bottleneck problems and
extract independent components
S. Klampfl, R. Legenstein, and W. Maass
Independent Component Analysis (or blind source separation) is assumed to be an
essential component of sensory processing in the brain and could provide a
less redundant representation about the external world. Another powerful
processing strategy is the optimization of internal representations according
to the information bottleneck method. This method would allow to extract
preferentially those components from high-dimensional sensory input streams
that are related to other information sources, such as internal predictions
or proprioceptive feedback. However there exists a lack of models that could
explain how spiking neurons could learn to execute either of these two
processing strategies. We show in this article how stochastically spiking
neurons with refractoriness could in principle learn in an unsupervised
manner to carry out both information bottleneck optimization and the
extraction of independent components. We derive suitable learning rules,
which extend the well known BCM-rule, from abstract information optimization
principles. These rules will simultaneously keep the firing rate of the
neuron within a biologically realistic range.
Reference: S. Klampfl, R. Legenstein, and W. Maass.
Spiking neurons can learn to solve information bottleneck problems and extract
Neural Computation, 21(4):911-959, 2009.