- Neal, R. (1992).
Connectionist learning of belief
networks.
Artificial Intelligence, 56: 71-113, 1992.
Remark: This paper introduces sigmoidal belief
networks and relates them to Boltzmann machines.
This talk should be the basis for a later talk on
deep belief networks.
Talks: 1 For the talk, one may skip everything
on noisy-or belief networks.
|
- G. E. Hinton, S. Osindero, and Y.-W. Teh.(2006)
A
fast learning algorithm for deep belief nets.
Neural Computation, 18:1527-1554, 2006.
The paper introduces Deep Belief Nets [DBNs] and how
such networks can be trained efficiently.
Talks: 2
-- (Sections 1-3): Intro; Complementary priors,
RBMs and Contrastive Divergence
-- (Sections 4-7): Greedy learning; Up-Down
Algorithm; Experiments on MNIST;
Conclusions
|
- M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L.
K. Saul. (1999)
An introduction to variational methods for graphical
models.
Machine Learning, 37:183-233, 1999.
Remark: This paper provides an introduction to
variational methods, that will be needed for the
training of Deep Boltzmann Machines.
For the talk, only Sections 1, 2 (maybe), 4, and 6 up
to 6.1 are relevant.
|
- R. Salakhutdinov and G.Hinton. (2012)
An Efficient Learning Procedure for Deep Boltzmann
Machines.
Neural Computation, 24(8), 1967-2006.
- - (Sections 1 and 2): Boltzmann machines, approximate
evaluation of the data-dependen and data-independent
distribtuions for learning.
- - (Section 3): Learning DBMs.
- - (Sections 4 and 5): Evaluating DBMs and
experimental results.
|
- R. Salakhutdinov, J. B. Tenenbaum, and A. Torralba
(2013)
Learning with hierarchical-deep models.
Pattern Analysis and Machine Intelligence, IEEE
Transactions on, 35(8), 1958-1971.
(shorter NIPS-Version)
-- Sections 1-3
-- Sections 4-6
|
- N. Srivastava and R. Salakhutdinov (2012).
Multimodal Learning with Deep Boltzmann
Machines.
NIPS 2012.
|
- V. Dumoulin, IJ Goodfellow, A. Courville, Y. Bengio
(2013).
On the
Challenges of Physical Implementations of
RBMs.
arXiv preprint arXiv: 1312.5258
|
- E. Neftci, S. Das, B. Pedroni, K. Kreutz-Delgado,
and G. Cauwenberghs (2013).
Event-Driven
Contrastive Divergence for Spiking Neuromorphic
Systems.
arXiv preprint arXiv: 1311.0966/.
|
- M. A. Petrovici, J. Bill, I. Bytschok, J. Schemmel,
and K. H. Meier.
Stochastic
inference with deterministic spiking
neurons.
arXiv :1311.3211 [q-bio.NC]
|
|
- Y. Bengio
Learning Deep Architectures for AI
Foundations and Trends in Machine Learning: Vol. 2 (1),
1-127, 2009.
|
|