Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
pdf Video Sup Mat video Code Project ppt
We describe the first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image. We estimate a full 3D mesh and show that 2D joints alone carry a surprising amount of information about body shape. The problem is challenging because of the complexity of the human body, articulation, occlusion, clothing, lighting, and the inherent ambiguity in inferring 3D from 2D. To solve this, we fi rst use a recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D body joint locations. We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints. We do so by minimizing an objective function that penalizes the error between the projected 3D model joints and detected 2D joints. Because SMPL captures correlations in human shape across the population, we are able to robustly fi t it to very little data. We further leverage the 3D model to prevent solutions that cause interpenetration. We evaluate our method, SMPLify, on the Leeds Sports, HumanEva, and Human3.6M datasets, showing superior pose accuracy with respect to the state of the art.
| Author(s): | Federica Bogo and Angjoo Kanazawa and Christoph Lassner and Peter Gehler and Javier Romero and Michael J. Black |
| Links: | |
| Book Title: | Computer Vision – ECCV 2016 |
| Pages: | 561--578 |
| Year: | 2016 |
| Month: | October |
| Series: | Lecture Notes in Computer Science |
| Publisher: | Springer International Publishing |
| Project(s): | |
| Bibtex Type: | Conference Paper (inproceedings) |
| Event Name: | 14th European Conference on Computer Vision |
| Event Place: | Amsterdam, The Netherlands |
| Electronic Archiving: | grant_archive |
BibTex
@inproceedings{Bogo:ECCV:2016,
title = {Keep it {SMPL}: Automatic Estimation of {3D} Human Pose and Shape from a Single Image},
booktitle = {Computer Vision -- ECCV 2016},
abstract = {We describe the first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image. We estimate a full 3D mesh and show that 2D joints alone carry a surprising amount of information about body shape. The problem is challenging because of the complexity of the human body, articulation, occlusion, clothing, lighting, and the inherent ambiguity in inferring 3D from 2D. To solve this, we fi rst use a recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D body joint locations. We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints. We do so by minimizing an objective function that penalizes the error between the projected 3D model joints and detected 2D joints. Because SMPL captures correlations in human shape across the population, we are able to robustly fi t it to very little data. We further leverage the 3D model to prevent solutions that cause interpenetration. We evaluate our method, SMPLify, on the Leeds Sports, HumanEva, and Human3.6M datasets, showing superior
pose accuracy with respect to the state of the art.},
pages = {561--578},
series = {Lecture Notes in Computer Science},
publisher = {Springer International Publishing},
month = oct,
year = {2016},
slug = {bogo-eccv-2016},
author = {Bogo, Federica and Kanazawa, Angjoo and Lassner, Christoph and Gehler, Peter and Romero, Javier and Black, Michael J.},
month_numeric = {10}
}