Seminar Computational Intelligence F (708.116)

WS 2013

Institut f�r Grundlagen der Informationsverarbeitung (708)

O.Univ.-Prof. Dr. Wolfgang Maass

Office hours: by appointment (via e-mail)

E-mail: legi@igi.tugraz.at
Homepage: https://www.tugraz.at/institute/igi/team/prof-legenstein/

Location: IGI-seminar room, Inffeldgasse 16b/I, 8010 Graz
Date: starting on October, 3rd 2013, every Thursday, 16.15 - 18.00

Our agreement was, to continue our seminar for phd students and other institute members in a similar way as last semester. We will meet this week on thursday, Oct. 3, at 16:15 in the seminar room to discuss your selection for paper presentations, and dates for talks.

In particular, we want to continue the light journal-club-like style of presentations of work by others, where the presenter has read the paper, and can perhaps show some figures from the paper, but otherwise mainly directs the discussion of the paper (assuming that all participants have looked before into the presented paper).

In addition, we want to continue the tradition of short informal presentations of own research results and problems on the blackboard. This semester we also want to schedule these presentations in advance, but no ppt slides should be shown (except for rare cases where figs that could not possibly be reproduced on the blackboard), and excessive preparation is not desired. If you aim at about 20min presentation time, this will in general be ok (leaving plenty of time for discussion).

Robert (strand computing, and particle filters), Florian (he should report the strategy of his work, and a bit on the preceding NIPS paper by Roelfsema, SHS (theory for CSP), Elmar (movement planning through neural sampling), David (motor control through neural sampling), Christoph (Ca-imaging data analysis), Dejan (learning BNs), Gernot (networks of sWTA's) are all working on interesting new topics, and should give a presentation before they get down to write the final version of a paper on the results (which suggests in most cases a presentation in Oct or Nov).

Either type of presentation should not last longer than 35 or at most 40 minutes.

In the following I have listed a few papers that either contain a new perspective, or contribute to our understanding of microcircuits (esp. role of inhibition, and how sensory input arrives there), and high level organization of brain computations:

D. Kvitsiani, S. Ranade, B. Hangya, H. Taniguchi, J. Z. Huang, and A. Kepecs. Distinct behavioural and network correlates of two interneuron types in prefrontal cortex. Nature, 498:363-366, 2013.

D. K�tzel, B. V. Zemelman, C. Buetfering, M. W�lfel, and G. Miesenb�ck. The columnar and laminar organization of inhibitory connections to neocortical excitatory cells. Nature Neuroscience, 14(1):100-107, 2011.

C. M. Constantinople and R. M. Bruno. Deep Cortical Layers Are Activated Directly by Thalamus. Science, 340:1591-1594, 2013.

J. R. Anderson, J. M. Fincham, Y. Qin, and A. Stocco. A central circuit of the mind. Trends in Cognitive Sciences, 12(4):136-143, 2008.
(This is a newer paper on ACT-R, which is the leading competitor to our stochastic approaches as primary brain engine for problem solving, including Sudoku solving:
Y. Qin, J. Y. Xiang, R. Wang, H. Zhou, K. Li, and N. Zhong. Neural bases for basic processes in heuristic problem solving: Take solving Sudoku puzzles as an example. PsyCh Journal, 1:101-117, 2012.

R. Stickgold and M. Walker. Sleep-dependent memory triage: evolving generalization through selective processing. Nature Neuroscience, 16(2):139-145, 2013.

-----------------------------------

In addition Robert has selected some options for papers on Reinforcement learning in the brain, especially on the interplay of model-based and model-free RL in the brain:

D. Lee, H. Seo, and W. M. Jung. Neural Basis of Reinforcement Learning and Decision Making. Annual Review Neuroscience, 35:287-308, 2012.
Review: Discusses model-free vs. model-based RL, with quite extensive references to brain areas.

B. B. Doll, D. A. Simon, and N. D. Daw. The ubiquity of model-based reinforcement learning. Curr Opin Neurobiol., 22:1075-1081, 2012.
Summarizes evidence for model-based RL in the brain.

Y. Niv. Reinforcement learning in the brain. Journal of Mathematical Psychology, 53:139-154, 2009.
Review: Also discusses model-free vs. model based RL with some interesting challenges and future directions. Maybe less suitable as it discusses model-free RL rather intensively and the model-based case less so.

H. Mushiake, N. Saito, K. Sakamoto, Y. Itoyama, and J. Tanji. Activity in the Lateral Prefrontal Cortex Reflects Multiple Steps of Future Events in Action Plans. Neuron, 50:631-641, 2006.
Experimental: I did not read through it yet, but it seems to present quite interesting data about planning-related activity in PFC.

C. Padoa-Schioppa and J. A. Assad. Neurons in the orbitofrontal cortex encode economic value. Nature, 441:223-226, 2006.
Experimental: Shows that neurons in OFC encode the value of goods (pretty short artilce).

J. Gl�scher, N. Daw, P. Dayan, and J. P. O'Doherty. States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66:585-595, 2010.
Experimental: Evidence for model-based RL and a "state prediction error" signal that could be useful to learn a model of the environment. (Daw has several papers on similar questions)

K. Samejima and K. Doya. Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops. Annals of the New York York Academy of Sciences, 1104:213-228, 2007.
Model/Experimental. Includes also a review of brain structures for various aspects of RL.

A. Solway and M. M. Botvinick. Goal-Directed Decision Making as Probabilistic Inference: A Computational Framework and Potential Neural Correlates. Psychological Review, 119(1):120-154, 2012.
Model: The Botvinick model links model-based RL to probabilistic inference (quite long paper).

K. Doya, K. Samejima, K. Katagiri, and M. Kawato. Multiple model-based reinforcement learning. Neural Computation, 14:1347-1369, 2002.
Model: I did not yet read through this but maybe an interesting model. At least one of the few models I found so far for model-based RL.

Talks:

30.10.2013 (Wednesday)
Elmar R�ckert and Dejan Pecevski

31.10.2013
Stefan Habenschuss and Zeno Jonke

21.11.2013
	Gernot Griesbacher and Florian Hubner

28.11.2013
Robert Legenstein

12.12.2013
Christoph Pokorny
Elmar R�ckert	on: Activity in the Lateral Prefrontal Cortex Reflects Multiple Steps of Future Events in Action Plans H Mushiake, N Saito, K Sakamoto, Y Itoyama, J Tanji - Neuron, 2006. pdf
19.12.2013
Octave Etard

09.01.2014
David Kappel	on: The columnar and laminar organization of inhibitory connections to neocortical excitatory cells K�tzel, D., Zemelman, B. V., Buetfering, C., W�lfel, M., & Miesenb�ck,G. (2010). Nature neuroscience. pdf


16.01.2014
Johannes Bill
Florian Hubner	on: Deep Cortical Layers Are Activated Directly by Thalamus Constantinople, C. M., & Bruno, R. M. (2013). Science, 340(6140), 1591-1594. pdf

23.01.2014
David Kappel
Octave Etard	on: New results regarding cortical amplification of thalamic input A mini-review of these is provided by: Y. Han and T. Mrsic-Flogel. A finely tuned cortical amplifier. Nature Neuroscience, 16(9):1166-1168, 2013. pdf

30.01.2014 15:15 pm
Dejan Pecevski	on: Probabilistic brains: knowns and unknowns A Pouget, JM Beck, WJ Ma, PE Latham - Nature Neuroscience, 2013. pdf



Stefan Habenschuss	on: Bayesian Just-So Stories in Psychology and Neuroscience. JS Bowers, CJ Davis - Psychological bulletin, 2012. pdf and How the Bayesians Got Their Beliefs (and What those Beliefs Actually Are): Comment on Bowers and Davis (2012).TL Griffiths, N Chater, D Norris, A Pouget - 2012. pdf