Seminar Computational Intelligence B (708.112)

Content of the seminar: Reinforcement Learning

In this seminar, we will discuss Reinforcement Learning in depth. Reinforcement Learning is a very important subfield of Machine Learning, where learning is not performed from explicit target labels, but from reward signals.
We will start with the basic concepts and algorithms. Our treatment of the topic will be based on a new book PDF.

Topics:

Basic Concepts an Algorithms

1. Ch.2 Multi-armed Bandits
2. Ch.3 Finite Markov Decision Processes
3. Ch.4 Dynamic Programming
4. Ch.5 Monte Carlo Methods
5. Ch.6 Temporal-Difference Learning
6. Ch.7 n-step Bootstrapping
7. Ch.8 Planning and Learning with Tabular Methods

Approximate Solutions

8. Ch.9 On-policy prediction with approximation
9. Ch.10 On-policy control with approximation
10. Ch.12 Eligibility traces
11. Ch.13 Policy Gradient Methods

Reinforcement Learning in Humans and Animals

12. Ch.14 Reinforcement Learning and Psychology
13. Ch.15 Reinforcement Learning and Neuroscience

Current Research Topics

14. Learning to play video games with Reinforcement Learning. Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529 [PDF] and Ch.16.5.
15. Learning to play board games with Reinforcement: Silver, David, et al. "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm." arXiv preprint arXiv:1712.01815 (2017). [PDF] and Ch.16.6. Learning
16. Learning to Learn: Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., ... & Botvinick, M. (2016). Learning to reinforcement learn. [PDF]

Talks should be no longer than 35 minutes, and they should be be clear, interesting and informative, rather than a reprint of the material. Select what parts of the material you want to present, and what not, and then present the selected material well (including definitions not given in the material: look them up on the web or if that is not successful, ask the seminar organizers). Often diagrams or figures are useful for a talk. on the other hand, giving in the talk numbers of references that are listed at the end is a no-no (a talk is an online process, not meant to be read). For the same reasons you can also quickly repeat earlier definitions or so if you suspect that the audience may not remember it.

Talks will be assigned at the first seminar meeting. Students are requested to have a quick glance at the topics prior to this meeting in order to determine their preferences. Note that the number of participants for this seminar will be limited. Preference will be given to students who

1. are / will write a Master's Thesis at the institute
2. are / will perform a Student's Project at the institute
3. have registered early.

General rules:

TALKS: