Learned graphical models for probabilistic planning provide a new class of movement primitives

Biological movement generation combines three interesting aspects: its modular organization in movement primitives, its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative movement primitive representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow movement primitives with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control methods within a movement primitive. The parametrization of the primitive is a graphical model that represents the dynamics and intrinsic cost func21 tion such that inference in this graphical model yields the control policy. We parametrize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement represen tation facilitates learning significantly and leads to better generalization to new task settings without re-learning.

Learned graphical models for probabilistic planning provide a new class of movement primitives

Abstract: