J. Kaiser, M. Hoff, A. Konle, J. C. V. Tieck, D. Kappel, D. Reichard,
A. Subramoney, R. Legenstein, A. Roennau, W. Maass, and R. Dillmann
The endeavor to understand the brain involves multiple collaborating research
fields. Classically, synaptic plasticity rules derived by theoretical
neuroscientists are evaluated in isolation on pattern classification tasks.
This contrasts with the biological brain which purpose is to control a body
in closed-loop. This paper contributes to bringing the fields of
computational neuroscience and robotics closer together by integrating
open-source software components from these two fields. The resulting
framework allows to evaluate the validity of biologically-plausibe plasticity
models in closed-loop robotics environments. We demonstrate this framework to
evaluate Synaptic Plasticity with Online REinforcement learning (SPORE), a
reward-learning rule based on synaptic sampling, on two visuomotor tasks:
reaching and lane following. We show that SPORE is capable of learning to
perform policies within the course of simulated hours for both tasks.
Provisional parameter explorations indicate that the learning rate and the
temperature driving the stochastic processes that govern synaptic learning
dynamics need to be regulated for performance improvements to be retained. We
conclude by discussing the recent deep reinforcement learning techniques
which would be beneficial to increase the functionality of SPORE on
visuomotor tasks.