Learning from demonstration
1997
Conference Paper
am
By now it is widely accepted that learning a task from scratch, i.e., without any prior knowledge, is a daunting undertaking. Humans, however, rarely attempt to learn from scratch. They extract initial biases as well as strategies how to approach a learning problem from instructions and/or demonstrations of other humans. For learning control, this paper investigates how learning from demonstration can be applied in the context of reinforcement learning. We consider priming the Q-function, the value function, the policy, and the model of the task dynamics as possible areas where demonstrations can speed up learning. In general nonlinear learning problems, only model-based reinforcement learning shows significant speed-up after a demonstration, while in the special case of linear quadratic regulator (LQR) problems, all methods profit from the demonstration. In an implementation of pole balancing on a complex anthropomorphic robot arm, we demonstrate that, when facing the complexities of real signal processing, model-based reinforcement learning offers the most robustness for LQR problems. Using the suggested methods, the robot learns pole balancing in just a single trial after a 30 second long demonstration of the human instructor.Â
Author(s): | Schaal, S. |
Book Title: | Advances in Neural Information Processing Systems 9 |
Pages: | 1040-1046 |
Year: | 1997 |
Editors: | Mozer, M. C.;Jordan, M.;Petsche, T. |
Publisher: | MIT Press |
Department(s): | Autonome Motorik |
Bibtex Type: | Conference Paper (inproceedings) |
Address: | Cambridge, MA |
Cross Ref: | p873 |
Note: | clmc |
URL: | http://www-clmc.usc.edu/publications/S/schaal-NIPS1997.pdf |
BibTex @inproceedings{Schaal_ANIPS_1997, title = {Learning from demonstration}, author = {Schaal, S.}, booktitle = {Advances in Neural Information Processing Systems 9}, pages = {1040-1046}, editors = {Mozer, M. C.;Jordan, M.;Petsche, T.}, publisher = {MIT Press}, address = {Cambridge, MA}, year = {1997}, note = {clmc}, doi = {}, crossref = {p873}, url = {http://www-clmc.usc.edu/publications/S/schaal-NIPS1997.pdf} } |