Analog neural nets with Gaussian or other common noise distributions
cannot recognize arbitrary regular languages
We consider recurrent analog neural netw where the output of each gate is
subject to gaussian noise or any other common noise distribution that is
nonzero on a sufficiently large part of the state-space. We show that many
regular languages cannot be recognized by networks of this type, and we give
a precise characterization of languges that can be recognized. This result
implies severe constraints on possibilities for constructing recurrent analog
neural nets that are robust against realistic types of analog noise. On the
other hand, we present a method for constructing feedforward analog neural
nets that are robust with regard to analog noise of this type.
Reference: W. Maass and E. Sontag.
Analog neural nets with Gaussian or other common noise distributions cannot
recognize arbitrary regular languages.
Neural Computation, 11:771-782, 1999.