Information Bottleneck Optimization and Independent Component Extraction with Spiking Neurons

S. Klampfl, R. Legenstein, and W. Maass


The extraction of statistically independent components from high-dimensional multi-sensory input streams is assumed to b e an essential component of sensory processing in the brain. Such independent component analysis (or blind source separat ion) could provide a less redundant representation of inform ation about the external world. Another powerful processing strategy is to extract preferentially those components from high-dimensional input streams that are related to other inf ormation sources, such as internal predictions or propriocep tive feedback. This strategy allows the optimization of inte rnal representation according to the information bottleneck method. However, concrete learning rules that implement thes e general unsupervised learning principles for spiking neuro ns are still missing. We show how both information bottlenec k optimization and the extraction of independent components can in principle be implemented with stochastically spiking neurons with refractoriness. The new learning rule that achi eves this is derived from abstract information optimization principles.

Reference: S. Klampfl, R. Legenstein, and W. Maass. Information bottleneck optimization and independent component extraction with spiking neurons. In Proc. of NIPS 2006, Advances in Neural Information Processing Systems, volume 19, pages 713-720. MIT Press, 2007.