 Neal, R. (1992).
Connectionist learning of belief
networks.
Artificial Intelligence, 56: 71113, 1992.
Remark: This paper introduces sigmoidal belief
networks and relates them to Boltzmann machines.
This talk should be the basis for a later talk on
deep belief networks.
Talks: 1 For the talk, one may skip everything
on noisyor belief networks.

 G. E. Hinton, S. Osindero, and Y.W. Teh.(2006)
A
fast learning algorithm for deep belief nets.
Neural Computation, 18:15271554, 2006.
The paper introduces Deep Belief Nets [DBNs] and how
such networks can be trained efficiently.
Talks: 2
 (Sections 13): Intro; Complementary priors,
RBMs and Contrastive Divergence
 (Sections 47): Greedy learning; UpDown
Algorithm; Experiments on MNIST;
Conclusions

 M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L.
K. Saul. (1999)
An introduction to variational methods for graphical
models.
Machine Learning, 37:183233, 1999.
Remark: This paper provides an introduction to
variational methods, that will be needed for the
training of Deep Boltzmann Machines.
For the talk, only Sections 1, 2 (maybe), 4, and 6 up
to 6.1 are relevant.

 R. Salakhutdinov and G.Hinton. (2012)
An Efficient Learning Procedure for Deep Boltzmann
Machines.
Neural Computation, 24(8), 19672006.
  (Sections 1 and 2): Boltzmann machines, approximate
evaluation of the datadependen and dataindependent
distribtuions for learning.
  (Section 3): Learning DBMs.
  (Sections 4 and 5): Evaluating DBMs and
experimental results.

 R. Salakhutdinov, J. B. Tenenbaum, and A. Torralba
(2013)
Learning with hierarchicaldeep models.
Pattern Analysis and Machine Intelligence, IEEE
Transactions on, 35(8), 19581971.
(shorter NIPSVersion)
 Sections 13
 Sections 46

 N. Srivastava and R. Salakhutdinov (2012).
Multimodal Learning with Deep Boltzmann
Machines.
NIPS 2012.

 V. Dumoulin, IJ Goodfellow, A. Courville, Y. Bengio
(2013).
On the
Challenges of Physical Implementations of
RBMs.
arXiv preprint arXiv: 1312.5258

 E. Neftci, S. Das, B. Pedroni, K. KreutzDelgado,
and G. Cauwenberghs (2013).
EventDriven
Contrastive Divergence for Spiking Neuromorphic
Systems.
arXiv preprint arXiv: 1311.0966/.

 M. A. Petrovici, J. Bill, I. Bytschok, J. Schemmel,
and K. H. Meier.
Stochastic
inference with deterministic spiking
neurons.
arXiv :1311.3211 [qbio.NC]


 Y. Bengio
Learning Deep Architectures for AI
Foundations and Trends in Machine Learning: Vol. 2 (1),
1127, 2009.

