PULSED NEURAL NETWORKS
by Wolfgang Maass and Christopher M. Bishop (eds.)
General Information about the Book
Table of Contents
Most practical applications of artificial neural networks are based on a computational model involving the propagation of continuous variables from one processing unit to the next. In recent years, data from neurobiological experiments have made it increasingly clear that biological neural networks, which communicate through pulses, use the timing of the pulses to transmit information and perform computation. This realization has stimulated significant research on pulsed neural networks, including theoretical analyses and model development, neurobiological modeling, and hardware implementation.
This book presents the complete spectrum of current research in pulsed neural networks and includes the most important work from many of the key scientists in the field. Terrence J. Sejnowski's foreword, "Neural Pulse Coding," presents an overview of the topic. The first half of the book consists of longer tutorial articles spanning neurobiology, theory, algorithms, and hardware. The second half contains a larger number of shorter research chapters that present more advanced concepts. The contributors use consistent notation and terminology throughout the book.
|The majority of artificial neural network models
are based on a computational paradigm involving the propagation of continuous
variables from one processing unit to the next. In recent years, however,
data from neurobiological experiments have made it increasingly clear that
biological neural networks, which communicate through pulses, use the timing
of these pulses to transmit information and to perform computation. This
realization has stimulated a significant growth of research activity in
the area of pulsed neural networks ranging from neurobiological modeling
and theoretical analyses, to algorithm development and hardware implementations.
Such research is motivated both by the desire to enhance our understanding
of information processing in biological networks, as well as by the goal
of developing new information processing technologies.
Our aim in producing this book has been to provide a comprehensive treatment of the field of pulsed neural networks, which will be accessible to researchers from diverse disciplines such as electrical engineering, signal processing, computer science, physics, and computational neuroscience. By virtue of its pedagogical emphasis, it will also find a place in many of the advanced undergraduate and graduate courses in neural networks now taught in many universities.
The Isaac Newton InstituteThis book originated from a two-day workshop entitled Pulsed Neural Networks that we organized in August 1997 at the Isaac Newton Institute for Mathematical Sciences in Cambridge1.The workshop formed part of the six-month Newton Institute program Neural Networks and Machine Learning, organized by Chris Bishop, David Haussler, Geoffrey Hinton, Mahesan Niranjan and Leslie Valiant. This research program was the largest international event of its kind to have taken place in the field of neural computing, and attracted several hundred participants for visits ranging from one or two weeks up to six months.
The workshop on Pulsed Neural Networks comprised two days of invited presentations by many of the foremost researchers in the field, and proved to be a very timely event. In view of the interdisciplinary nature of this subject, the workshop included a number of tutorials that introduced pulsed neural networks from the point of view of different disciplines. As a result of the success of the workshop, there was considerable enthusiasm to capture the highlights of the meeting in book form and thereby make the workshop contributions, including both tutorials and research presentations, accessible to a much wider audience. All contributions were rewritten to take into account the special context of this book, and to use consistent terminology and notation across the different disciplines. We hope this book will convey some of the excitement of the workshop and of the field of pulsed neural networks.
Overview of the BookThe Foreword by Terry Sejnowski sets the stage for the book. The core of the book consists of three parts. The first part (Basic Concepts and Models) comprises four tutorial chapters. The tutorial Spiking Neurons (Chapter 1) by Wulfram Gerstner introduces the neurophysiological background and motivations for computing with pulses. It discusses a simple mathematical model for a spiking neuron, the spike response model, that provides the basis for later chapters. The tutorial Computing with Spiking Neurons (Chapter 2) by Wolfgang Maass analyzes the computational power of networks of spiking neurons, and compares them with traditional neural network models. Hardware implementations of pulsed neural nets are discussed in the tutorial Pulsed-Based Computations in VLSI Neural Networks (Chapter 3) by Alan Murray. The tutorial Encoding Information in Neural Activity (Chapter 4) by Michael Recce surveys results about the way in which biological systems encode information in spatial-temporal patterns of pulses.
The chapters in the second part of the book (Implementations) review a number of options for implementing pulsed neural nets in electronic hardware. Chapters 5 to 8 discuss approaches, and first results, for implementing artificial pulsed neural nets in analog VLSI. Chapter 9 reviews the state of the art regarding digital simulations of pulsed neural nets.
The third part of the book (Design and Analysis of Pulsed Neural Systems) surveys current research on the design and analysis of pulsed neural networks, in both biological and artificial systems.
Each of the chapters in the second and third part should be comprehensible to anyone who has worked through the four tutorials in the first part of the book. Together these chapters constitute a survey of current research issues across all aspects of pulsed neural networks including mathematical analyses, algorithms, hardware and software implementations, and neurobiology.
Christopher M. Bishop and Wolfgang Maass
1Further information about the Isaac Newton Institute can be found at http://www.newton.cam.ac.uk.
Christopher M. Bishop Preface Microsoft Research, Cambridge Cambridge CB2 3NH, England, UK email@example.com http://research.microsoft.com/~cmbishop/
Peter S. Burge Chapter 13 Department of Computer Science Royal Holloway, University of London Egham, England, UK firstname.lastname@example.org
Max R. van Daalen Chapter 13 Department of Computer Science Royal Holloway, University of London Egham, England, UK email@example.com
Stephen R. Deiss Chapter 6 Applied Neurodynamics Encinitas, CA, 92024-5354, USA firstname.lastname@example.org
Rodney J. Douglas Chapter 6 Institut für Neuroinformatik Universität Zürich & ETH Zürich Zürich, Switzerland email@example.com
John G. Elias Chapter 5 Department of Electrical and Computer Engineering University of Delaware Newark, Delaware 19716, USA firstname.lastname@example.org
Wulfram Gerstner Chapters 1, 10, 14 Center for Neuromimetic Systems Swiss Federal Institute of Technology, EPFL CH-1015 Lausanne, Switzerland Wulfram.Gerstner@di.epfl.ch
Alister Hamilton Chapter 8 Dept. of Electrical Engineering University of Edinburgh Edinburgh, Scotland, UK Alister.Hamilton@ee.ed.ac.uk
J. Leo van Hemmen Chapter 14 Physik Department, TU München D-85747 Garching bei München München, Germany Leo.van.Hemmen@Physik.TU-München.de
David Horn Chapter 11 School of Physics and Astronomy Tel Aviv University Tel Aviv, Israel email@example.com
Axel Jahnke Chapter 9 Institut of Microelectronics TU Berlin Berlin, Germany firstname.lastname@example.org
Richard Kempter Chapter 14 Institut für Theoretische Physik Physik-Department der TU München München, Germany Richard.Kempter@Physik.TU-Muenchen.DE
Wolfgang Maass Preface, Chapters 2, 12 Institute of Theoretical Computer Science Technische Universität Graz A-8010 Graz, Austria email@example.com
Alessandro Mortara Chapter 7 Advanced Microelectronics Division Centre Suisse d'Electronique et de Microtechnique Neuchatel, Switzerland firstname.lastname@example.org
Alan F. Murray Chapter 3 Dept. of Electrical Engineering University of Edinburgh Edinburgh, EH9 3JL., England, UK Alan.Murray@ee.ed.ac.uk
David P. M. Northmore Chapter 5 Department of Psychology University of Delaware Newark, Delaware 19716, USA email@example.com
Irit Opher Chapter 11 School of Physics and Astronomy Tel Aviv University Tel Aviv, Israel firstname.lastname@example.org
Kostas A. Papathanasiou Chapter 8 Department of Electrical Engineering University of Edinburgh Edinburgh, Scotland, UK Kostas.Papathanasiou@ee.ed.ac.uk
Michael Recce Chapter 4 Department of Computer and Information Science New Jersey Institute of Technology Newark, NJ 07102, USA email@example.com
Barry J. P. Rising Chapter 13 Department of Computer Science Royal Holloway, University of London Egham, England, UK firstname.lastname@example.org
Ulrich Roth Chapter 9 Institut of Microelectronics TU Berlin Berlin, Germany email@example.com
Tim Schönauer Chapter 9 Institut of Microelectronics TU Berlin Berlin, Germany firstname.lastname@example.org
Terrence J. Sejnowski Foreword The Salk Institute La Jolla, CA 92037, USA email@example.com
John S. Shawe-Taylor Chapter 13 Department of Computer Science Royal Holloway, University of London Egham, UK firstname.lastname@example.org
Philippe Venier Chapter 7 Advanced Microelectronics Division Centre Suisse d'Electronique et de Microtechnique Neuchatel, Switzerland email@example.com
Hermann Wagner Chapter 14 Institut für Biologie Lehrstuhl für Zoologie/Tier- physiologie RWTH Aachen D-52074 Aachen, Germany firstname.lastname@example.org
Adrian M. Whatley Chapter 6 Institut für Neuroinformatik Universität Zürich & ETH Zürich Zürich, Switzerland email@example.com
Anthony M. Zador Chapter 12 The Salk Institute La Jolla, CA 92037, USA firstname.lastname@example.org
Foreword by Terrence J. Sejnowski
Contributors to the book