Machine Learning A

Machine Learning A, WS 2009
Institut f�r Grundlagen der Informationsverarbeitung (7080)	last modifications:

Course content:

This course provides an overview of the state-of-the-art in the areas of pattern recognition and probabilistic inference, and a thorough understanding of the most successful methods in those areas.

After successful completion of the course, the students are familiar with stragtegies to solve conceptual and theoretical problems in machine learning. They have the ability to apply state-of-the-art algorithms to real-world problems and are familiar with the advantages and disadvantages of different algorithms.

Covered topics:

Inference from uncertain facts (Bayesian networks etc)
Kernel methods
Boosting
Mixtures of experts

News

Watch out for news here.

14.09.09

Homepage installed. If you have any questions, send an e-mail.

People Involved

This course is being organized by Institut für Grundlagen der Informationsverarbeitung, Inffeldgasse 16b/1. Stock, A-8010 Graz.

Lecturer/Instructors

O.Univ.-Prof. Dr. Wolfgang Maass

Office hours: on appointment

e-mail: maass@igi.tugraz.at
Homepage: https://igi-web.tugraz.at/people/maass/

Mag. Dr. Stefan Häusler

Office hours: on appointment

e-mail: haeusler@igi.tugraz.at

Office

Angelika Zehetner
Office hours:: Mo - Fr, 08:00- 12:00 am
email: angelika@igi.TUGraz.at
Tel.: 873 5812
Homepage: www.igi.tugraz.at

If you have any questions or problems, please do not hesitate to contact one of the above persons.

Time and Location:

Lecture

Time: Monday, 13.15-14.45
First lecture: Monday, 12.10.2009
Location: Siemens TS Hörsaal (i 11), Inffeldgasse 16b

Exercises

Time: Dienstag, 12.15-13:00
First exercise: Tuesday, 20.10.2009
Location: i 12, Inffeldgasse 16b

Literature:

Applications for Topic Models: Text and Beyond, NIPS Workshop: Dec 11, 2009, Whistler, Canada.
J. C. Niebles, W. Hongcheng, and L. Fei-Fei: Unsupervised Learning of Human Action Categories Using Spatial-TemporalWords, Int J Comput Vis (2008) 79: 299–318.
M. Steyvers, and T. Griffiths: Probabilistic Topic Models, In: In T. Landauer, D McNamara, S. Dennis, and W. Kintsch (eds), Latent Semantic Analysis: A Road to Meaning. Laurence Erlbaum.
T. Griffiths, and M. Steyvers: Finding scientific topics, PNAS, 101: 5228–5235, (2004).
L. Fei-Fei, and P. Perona: A Bayesian Hierarchical Model for Learning Natural Scene Categories, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2: 524 - 531, (2005).
Recognizing the Gist of a Scene , Visual Cognition Laboratory, Kansas State University , US.
National Science Foundation Workshop Report: Future Challenges for the Science and Engineering of Learning, July 23-25 (2007), Arlington, Virginia, US.
T. G. Dietterich, G. Bakiri: Solving Multiclass Learning Problems via Error-Correcting Output Codes, Journal of Artificial Intelligence Research 2 (1995), 263-286.
Y. Freund, R.E. Schapire: A short introduction to boosting,J. Japan. Soc. for Artif. Intel. (1999), 771-780.
H. Grabner, H. Bischof: On-line Boosting and Vision.
R. Schapire: Theory and Applications of Boosting.
Chapter: Mixture Models and EM.
B.J. Frey and D. Dueck: Clustering by Passing Messages Between Data Points, Science (2007), 315:972.
F.R. Kschischang, B.J. Frey, and H.A. Loeliger: Factor Graphs and the Sum-Product Algorithm,IEEE Transactions on Information Theory, Vol. 47, No. 2, 2001.
J. Pearl: Probabilistic Reasoning in Intelligent Systems: Network of Plausible Inference, Morgan Kaufmann, 1988.
C. Bishop: Pattern Recognition and Machine Learning, Springer Verlag, 2006. In der Lehrbuchsammlung vorhanden (20 Stück)
T. M. Mitchell: Machine Learning, McGraw-Hill, New York, 1997. In der Lehrbuchsammlung vorhanden (10 Stück)
B. Schölkopf, A. J. Smola: Learning with Kernels, MIT Press, Cambridge MA, 2002.In der Lehrbuchsammlung vorhanden (10 Stück)
R. O. Duda, P.E. Hart, D. G. Stork: Pattern Classification, 2nd edition, John Wiley & Sons, 2001. In der Lehrbuchsammlung vorhanden (20 Stück)
F. V. Jensen, T. D. Nielsen: Bayesian Networks and Decision Graphs, 2nd edition, Springer-Verlag, New York, 2007. In der Lehrbuchsammlung vorhanden (3 Stück)
I. H. Witten, E. Frank: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufmann, 1999.
J.R. Quinlan: C4.5: Programms for Machine Learning, Morgan Kaufmann, 1992.
V. N. Vapnik: Statistical Learning Theory, John Wiley & Sons, New York, 1998.

Supporting material:

Literature | Software | Lecture slides | Tutorials

Literature

Applications for Topic Models: Text and Beyond, NIPS Workshop: Dec 11, 2009, Whistler, Canada.
J. C. Niebles, W. Hongcheng, and L. Fei-Fei: Unsupervised Learning of Human Action Categories Using Spatial-TemporalWords, Int J Comput Vis (2008) 79: 299–318.
M. Steyvers, and T. Griffiths: Probabilistic Topic Models, In: In T. Landauer, D McNamara, S. Dennis, and W. Kintsch (eds), Latent Semantic Analysis: A Road to Meaning. Laurence Erlbaum.
T. Griffiths, and M. Steyvers: Finding scientific topics, PNAS, 101: 5228–5235, (2004).
L. Fei-Fei, and P. Perona: A Bayesian Hierarchical Model for Learning Natural Scene Categories, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2: 524 - 531, (2005).
Recognizing the Gist of a Scene , Visual Cognition Laboratory, Kansas State University , US.
O. Pujol, P. Radeva: Discriminant ECOC: A Heuristic Method for Application Dependent Design of Error Correcting Output Codes IEEE Transaction on pattern analysis and machine learningn intelligence (2006), Vol. 28, No. 6
C. M. Bishop: Pattern Recognition and Machine Learning
P. Domingos, M. Pazzani: On the Optimality of the Simple Bayesian Classifier under Zero-One Loss
A.Y. Ng, M.I. Jordan: On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes
Y. Freund, R.E. Schapire: A short introduction to boosting
P. Viola, M. Jones: Robust real-time object detection
T.G. Dietterich: Ensemble Methods in Machine Learning
R.E. Shapire, Y. Freund, P. Bartlett: Boosting the margin: A new explanation for the effectiveness of voting methods
R. Meir, G. Raetsch: An Introduction to Boosting and Leveraging (Theorem 2 was quoted in the course)
N.C. Oza, S. Russell: Online Bagging and Boosting

Z. Ghahramani: Graphical Models: Parameter Learning
S. Borman: The Expectation Maximization Algorithm: A short Tutorial
M.I. Jordan, R.A. Jacobs: Hierarchical Mixtures of Experts and the EM Algorithm
M. Weber, M. Welling, P. Perona: Unsupervised Learning of Models for Recognition
J.A. Bilmes: A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models

Z. Ghahramani: Unsupervised Learning
D. Heckerman: A tutorial on learning with Bayesian networks
T.G. Dietterich, F. Draft: Machine Learning Research: Four current directions
M.I. Jordan: Graphical Models
M.I. Jordan, Y. Weiss: Graphical Models: Probabilistic Inference

B. Schölkopf: Introduction to Kernel Methods
N. Cristianini: Kernel Methods for Pattern Analysis

Software

WEKA

Slides of the lectures

Lecture	Date	Topic	Material
1	19.10.2009	Bayesche Netzwerke	Slides
2	11.1.2010	Topic Models	ppt
3	18.1.2010	Application of Topic Models	ppt

Tutorials

Affinity propagation

Webpage of Brendan J. Frey and Delbert Dueck

Information about Boosting

www.boosting.org

Applets

AdaBoost Applet (by Yoav Freund)
Classification Applet (by Gilad Mishne)

EM Algorithm for Mixture of Gaussians (by S. Akaho and O. Michel)

Search engines for scientific papers:

CiteSeer
You can search for keywords and authors of papers in the field of computer science. You can follow links to related articles to expand your search.
CiteSeer Computer Science Directory
Search engine for different topics (e.g. machine learning). Links to the most cited articles in different areas.
Google Scholar
Search engine Google in particular for scientific papers.
NIPS
NIPS is perhaps the most important (and very selective) annual conference in the field of subsymbolic machine learning. You can search for keywords and authors.

Journals

Journal of Machine Learning Research
One of the most important journals for machine learning. All articles are available online.
Machine Learning
The second important journal for machine learning, with many online articles.

Conferences

NIPS
Neural Information Processing Systems Conference.
ICML 2005
Most important annual conference for machine learning (this conference emerged from an AI background and therefore has more work on non-analog methods compared to NIPS. (see also ICML 2006)
ECML 2005
Annual european conference for machine learning.

Software

WEKA
Machine Learning Software in Java of the University of Waikato in New Zealand.
- Documentation
- Tutorial (chapter 8 of the book Data Mining by Witten and Frank).
- WEKA Experiment Environment
MLC++
Machine Learning Library in C++ by Silicon Graphics
The Spider
Machine Learning Environment for Matlab of the MPI for biological cybernetics in Tübingen.

Further links

CVOnline: Visual Learning
Information about machine learning in computer vision. Part of the CVOnline, an online compendium for computer vision.
www.kernel-machines.org
Information about Support Vector Machines and Kernel Methods.
UCI Machine Learning Repository
Collection of machine learning datasets.
The Machine Learning Network