Human Pose, Shape and Action
3D Pose from Images
2D Pose from Images
Beyond Motion Capture
Action and Behavior
Body Perception
Body Applications
Pose and Motion Priors
Clothing Models (2011-2015)
Reflectance Filtering
Learning on Manifolds
Markerless Animal Motion Capture
Multi-Camera Capture
2D Pose from Optical Flow
Body Perception
Neural Prosthetics and Decoding
Part-based Body Models
Intrinsic Depth
Lie Bodies
Layers, Time and Segmentation
Understanding Action Recognition (JHMDB)
Intrinsic Video
Intrinsic Images
Action Recognition with Tracking
Neural Control of Grasping
Flowing Puppets
Faces
Deformable Structures
Model-based Anthropometry
Modeling 3D Human Breathing
Optical flow in the LGN
FlowCap
Smooth Loops from Unconstrained Video
PCA Flow
Efficient and Scalable Inference
Motion Blur in Layers
Facade Segmentation
Smooth Metric Learning
Robust PCA
3D Recognition
Object Detection
Associative Skill Memories
One challenge towards autonomous manipulation is to cope with uncertainty and noise in the sensory-motor system of the robot and the environment. We argue that such a system requires to close feedback control loops in novel ways, using predictive models and leveraging previous experiences.
Manipulation tasks often decompose into a sequence of skills, where each skill can be seen as a stereotypical movement, with respect to its goal frame. Assuming that movement dictates sensory feedback, sensory traces can be associated with the movement primitive (e.g. encoded as DMPs), forming Associative Skill Memories (ASMs) [].
ASMs allow to learn predictive models, which can be used to cope with uncertainty and noise in the perceptual system of the robots. ASMs have been successfully used to address several important issues towards an autonomous manipulation platform.
- Having a general skill library which allows to e.g. grasp an object robustly, the sequence of skills has to be determined based on the sensory feedback of the execution. [] showed how to incrementally determine a correct skill sequence despite perceptual errors using ASMs.
- Errors both in the perceptual and action system are inevitable. Predictive models learned from ASMs allow to predict failures in an online fashion [
]. - For robust manipulation errors have to be detected online and possible successor skills selected in real time. In [] this problem is formulated as a data-driven online decision making problem, integration visual, acustic, and haptic sensors, as well as the system state to make real time decisions using the sensory information provided by the ASMs.
Interesting open research questions related to ASMs and manipulations are: Automatic skill decomposition; Learning good representation from unsupervised sensory information; Better failure incorporation.
[]
[]
Members
Publications