Efficient Temporal Processing with Biologically Realistic Dynamic Synapses

T. Natschlaeger, W. Maass, and A. Zador

Abstract:

Experimental data show that biological synapses behave quite differently from the symbolic synapses in common artificial neural network models. Biological synapses are dynamic, i.e., their ``weight'' changes on a short time scale by several hundred percent in dependence of the past input to the synapse. Here we describe a general model of computation that exploits dynamic synapses, and use a backpropagation-like algorithm to adjust the synaptic parameters. We show that such gradient descent suffices to approximate a given quadratic filter by a rather small neural system with dynamic synapses. We demonstrate that with this approach the nonlinear filter considered in (Back and Tsoi, 1993) can be approximated slightly better than by their model. Our numerical results are complemented by theoretical analysis which show that even with just a single hidden layer such networks can approximate a surprisingly large class of nonlinear filters: all filters that can be characterized by Volterra series. This result is robust with regard to various changes in the model for synaptic dynamics.

Reference: T. Natschlaeger, W. Maass, and A. Zador. Efficient temporal processing with biologically realistic dynamic synapses. Network: Computation in Neural Systems, 12:75-87, 2001.