A. Rao, P. Plank, A. Wild, and W. Maass
In spite of intensive efforts it has remained an open problem to what extent
current Artificial Intelligence (AI) methods that employ Deep Neural Networks
(DNNs) can be implemented more energy-efficiently on spike-based neuromorphic
hardware. This holds in particular for AI methods that solve sequence
processing tasks, a primary application target for spike-based neuromorphic
hardware. One difficulty is that DNNs for such tasks typically employ Long
Short-Term Memory (LSTM) units.Yet an efficient emulation of these units in
spike-based hardware has been missing. We present a biologically inspired
solution that solves this problem. This solution enables us to implement a
major class of DNNs for sequence processing tasks such as time series
classification and question answering with substantial energy savings on
neuromorphic hardware. In fact, the Relational Network for reasoning about
relations between objects that we use for question answering is the first
example of a large DNN that carries out a sequence processing task with
substantial energy-saving on neuromorphic hardware.