Institute Homepage
Institute Homepage DE Sign In

Back

Autonomous Learning Miscellaneous 2025

Emergence of natural and robust bipedal walking by learning from biologically plausible objectives

Thumb ticker sm 28399a34 d01a 4482 b179 0eb4c2178dfb
Empirical Inference
Pierre Schumacher
  • Doctoral Researcher
Thumb ticker sm georg 2018 crop small
Empirical Inference, Autonomous Learning
Georg Martius
Senior Research Scientist

Humans show unparalleled ability when maneuvering diverse terrains. While reinforcement learning (RL) has shown great promise for musculoskeletal simulation in the development of robust controllers, complex behaviors are only achievable under extensive use of motion data. We demonstrate that the combination of a recent RL algorithm with a biologically plausible reward is capable of learning controllers for 4 different musculoskeletal models and achieves locomotion with up to 90 muscles without demonstrations. Our controllers generalize to diverse and unseen terrains, while only a single adaptive objective function is needed for training. We validate our findings on four models in two different simulators. The RL agents perform robustly with complex 3D models, where reflex-controllers are difficult to apply, and produce close-to-natural motion. This is a first step for the motor control, biomechanics, and rehabilitation communities to generate complex human movements with RL, without using motion data or simple unrepresentative models.

Author(s): Pierre Schumacher and Thomas Geijtenbeek and Vittorio Caggiano and Vikash Kumar and Syn Schmitt and Georg Martius and Daniel F.B. Haeufle
Journal: iScience
Volume: 28
Number (issue): 4
Pages: 112203
Year: 2025
Month: April
Day: 18
Project(s):
BibTeX Type: Miscellaneous (misc)
DOI: https://doi.org/10.1016/j.isci.2025.112203
Electronic Archiving: grant_archive
Eprint: arXiv 2309.02976
State: Published
URL: https://arxiv.org/abs/2309.02976

BibTeX 

@misc{schumacher2024:NaturalAndRobustWalking,
  title = {Emergence of natural and robust bipedal walking by learning from biologically plausible objectives},
  journal = {iScience},
  abstract = {Humans show unparalleled ability when maneuvering diverse terrains. While reinforcement learning (RL) has shown great promise for musculoskeletal simulation in the development of robust controllers, complex behaviors are only achievable under extensive use of motion data. We demonstrate that the combination of a recent RL algorithm with a biologically plausible reward is capable of learning controllers for 4 different musculoskeletal models and achieves locomotion with up to 90 muscles without demonstrations. Our controllers generalize to diverse and unseen terrains, while only a single adaptive objective function is needed for training. We validate our findings on four models in two different simulators. The RL agents perform robustly with complex 3D models, where reflex-controllers are difficult to apply, and produce close-to-natural motion. This is a first step for the motor control, biomechanics, and rehabilitation communities to generate complex human movements with RL, without using motion data or simple unrepresentative models.},
  volume = {28},
  number = {4},
  pages = {112203},
  month = apr,
  year = {2025},
  author = {Schumacher, Pierre and Geijtenbeek, Thomas and Caggiano, Vittorio and Kumar, Vikash and Schmitt, Syn and Martius, Georg and Haeufle, Daniel F.B.},
  doi = {https://doi.org/10.1016/j.isci.2025.112203},
  eprint = {arXiv 2309.02976},
  url = {https://arxiv.org/abs/2309.02976},
  month_numeric = {4}
}