Empirical Inference Conference Paper 2011

Reinforcement Learning to adjust Robot Movements to New Situations

no image
Empirical Inference
Thumb ticker sm 12009745 10103538825457245 7502907506146263960 n
Empirical Inference
Research Group Leader

Many complex robot motor skills can be represented using elementary movements, and there exist efficient techniques for learning parametrized motor plans using demonstrations and self-improvement. However, in many cases, the robot currently needs to learn a new elementary movement even if a parametrized motor plan exists that covers a similar, related situation. Clearly, a method is needed that modulates the elementary movement through the meta-parameters of its representation. In this paper, we show how to learn such mappings from circumstances to meta-parameters using reinforcement learning.We introduce an appropriate reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. We compare this algorithm to several previous methods on a toy example and show that it performs well in comparison to standard algorithms. Subsequently, we show two robot applications of the presented setup; i.e., the generalization of throwing movements in darts, and of hitting movements in table tennis. We show that both tasks can be learned successfully using simulated and real robots.

Author(s): Kober, J. and Oztop, E. and Peters, J.
Book Title: Robotics: Science and Systems VI
Journal: Robotics: Science and Systems VI
Pages: 33-40
Year: 2011
Month: September
Day: 0
Editors: Matsuoka, Y. , H. F. Durrant-Whyte, J. Neira
Publisher: MIT Press
Bibtex Type: Conference Paper (inproceedings)
Address: Cambridge, MA, USA
Event Name: 2010 Robotics: Science and Systems Conference (RSS 2010)
Event Place: Zaragoza, Spain
Digital: 0
Electronic Archiving: grant_archive
ISBN: 978-0-262-51681-5
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik
Links:

BibTex

@inproceedings{6438,
  title = {Reinforcement Learning to adjust Robot Movements to New Situations},
  journal = {Robotics: Science and Systems VI},
  booktitle = {Robotics: Science and Systems VI},
  abstract = {Many complex robot motor skills can be represented using elementary movements, and there exist efficient
  techniques for learning parametrized motor plans using demonstrations and self-improvement. However, in
  many cases, the robot currently needs to learn a new elementary movement even if a parametrized motor
  plan exists that covers a similar, related situation. Clearly, a method is needed that modulates the elementary
  movement through the meta-parameters of its representation. In this paper, we show how to learn such
  mappings from circumstances to meta-parameters using reinforcement learning.We introduce an appropriate
  reinforcement learning algorithm based on a kernelized version of the reward-weighted regression. We
  compare this algorithm to several previous methods on a toy example and show that it performs well in
  comparison to standard algorithms. Subsequently, we show two robot applications of the presented setup;
  i.e., the generalization of throwing movements in darts, and of hitting movements in table tennis. We show
  that both tasks can be learned successfully using simulated and real robots.},
  pages = {33-40},
  editors = {Matsuoka, Y. , H. F. Durrant-Whyte, J. Neira},
  publisher = {MIT Press},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = sep,
  year = {2011},
  slug = {6438},
  author = {Kober, J. and Oztop, E. and Peters, J.},
  month_numeric = {9}
}