Institute Homepage
Institute Homepage Sign In

Back

Research Overview

Inferring and exploiting contact

Generative Proxemics: A Prior for 3D Social Interaction from Images

BITE -- Dog Shape and Pose from an Image

HOLD -- inferring 3D hand and object shape from video

MOVER -- Reconstructing 3D Scenes and People using Interaction

Datasets for understanding humans and animals

The Poses for Equine Research Dataset (PFERD)

BEAT2 Dataset for Holistic Co-Speech Gesture Generation

ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

The BioAMASS Dataset

OpenCapBench dataset

Human health and the 3D body

Body Shape Models in Treating Anorexia Nervosa

Customized Bone Plants for Humerus Shaft Fractures

Reconstructing Signing Avatars From Video Using Linguistic Priors

The AI animator

HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles

Gaussian Garments

PuzzleAvatar: Assembling 3D Avatars from Personal Albums

FLARE: Fast Learning of Animatable and Relightable Mesh Avatars

Language, Vision, and World Models

AWOL: Analysis WithOut synthesis using Language

Re-Thinking Inverse Graphics with Large Language Models

TeCH: Text-guided Reconstruction of Clothed Humans

Human pose, shape, and motion capture

WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion

3D Human Pose Estimation via Intuitive Physics

Accurate 3D Body Shape Regression using Metric and Semantic Attributes

BEV

Generating human motion

Generating Human Interaction Motions in Scenes with Text Control

TEMOS: Generating Diverse Human Motions from Text

EMAGE: Full-body Gestures from Audio

TEACH: Temporal Action Compositions for 3D Humans

Robot Perception Group

AirCap: 3D Motion Capture

AirCap: Perception-Based Control

AirCapRL: Aerial Motion Capture Using Deep RL

Data Team

Lab Tours and Public Outreach

Collecting Data - From the Idea to the Publication

Capture Technologies Setup

Completed Projects

Human Pose, Shape and Action

3D Pose from Images

2D Pose from Images

Beyond Motion Capture

Action and Behavior

Body Perception

Body Applications

Pose and Motion Priors

Clothing Models (2011-2015)

Reflectance Filtering

Learning on Manifolds

Markerless Animal Motion Capture

Multi-Camera Capture

2D Pose from Optical Flow

Body Perception

Neural Prosthetics and Decoding

Part-based Body Models

Intrinsic Depth

Lie Bodies

Layers, Time and Segmentation

Understanding Action Recognition (JHMDB)

Intrinsic Video

Intrinsic Images

Action Recognition with Tracking

Neural Control of Grasping

Flowing Puppets

Faces

Deformable Structures

Model-based Anthropometry

Modeling 3D Human Breathing

Optical flow in the LGN

FlowCap

Smooth Loops from Unconstrained Video

PCA Flow

Efficient and Scalable Inference

Motion Blur in Layers

Facade Segmentation

Smooth Metric Learning

Robust PCA

3D Recognition

Object Detection

Clothing Capture and Modeling

Modeling Human Movement

Action and Behavior

Differentiable Rendering

Image Segmentation and Semantics

Groups and Crowds

Learning from Synthetic Data

Learning Deep Representations of 3D

Efficient and Scalable Inference

Hierarchical Graphs for Generalized Modelling of Clothing Dynamics

Learning to Resolve Intersections in Neural Multi-Garment Simulations

Reconstructing Simulation-Ready Clothing with Photo-Realistic Appearance from Multi-View Video

Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles

Human Hair Reconstruction with Strand-Aligned 3D Gaussians

Joker: Conditional 3D Head Synthesis with Extreme Facial Expressions

Goal Driven Motion Generation

Perceiving Systems Members Publications

Model-based Anthropometry

Research photo pipeline6
We predict anthropometric measurements from multiple poses using a model-based approach. We start with a database of 3D scans in multiple poses (standing, sitting) with corresponding anthropometric measurements (CAESAR). First, we register the scans using prior knowledge about human body shape (a human body model). Second, we extract shape features. We consider local features, such as body circumferences and limb lengths, as well as global features, such as statistics on edge lengths and triangle deformations of the registered meshes. Third, we learn optimal features for predicting each measurement.
Additional

Extracting anthropometric or tailoring measurements from 3D human body scans is important for applications such as virtual try-on, custon clothing, and online sizing. Existing commercial solutions identify anatomical landmarks on high-resolution 3D scans and then compute distances or circumferences on the scan. Landmark detection is sensitive to acquisition noise (e.g. holes) and these methods require subjects to adopt a specific pose. In contrast, we propose a solution that we call model-based anthropometry. We fit a deformable 3D body model to scan data in one or more poses; this model-based fitting is robust to scan noise. This brings the scan into registration iwth a database of registered body scans. Then, we extract features from the registered model (rather than from the scan); these include, limb lengths, circumferences, and statistical features of global shape. Finally, we learn a mapping from these features to measurements using regularized linear regression. We perform an extensive evaluation using the CAESAR dataset and demonstrate that the accuracy of our method outperforms state-of-art methods.

Additional

Details on measurement prediction accuracy:

FEMALE
results (data)

MALE
results (data)

Members

Thumb ticker sm thumb img 1084 2
Perceiving Systems
Aggeliki Tsoli
Thumb ticker sm headshot2021
Perceiving Systems
Michael Black
Director
Thumb ticker sm thumb img 1090
Perceiving Systems
Matthew Loper

Publications

Perceiving Systems Conference Paper Model-based Anthropometry: Predicting Measurements from 3D Human Scans in Multiple Poses Tsoli, A., Loper, M., Black, M. J. In Proceedings Winter Conference on Applications of Computer Vision, 83-90, IEEE , IEEE Winter Conference on Applications of Computer Vision (WACV) , March 2014
Extracting anthropometric or tailoring measurements from 3D human body scans is important for applications such as virtual try-on, custom clothing, and online sizing. Existing commercial solutions identify anatomical landmarks on high-resolution 3D scans and then compute distances or circumferences on the scan. Landmark detection is sensitive to acquisition noise (e.g. holes) and these methods require subjects to adopt a specific pose. In contrast, we propose a solution we call model-based anthropometry. We fit a deformable 3D body model to scan data in one or more poses; this model-based fitting is robust to scan noise. This brings the scan into registration with a database of registered body scans. Then, we extract features from the registered model (rather than from the scan); these include, limb lengths, circumferences, and statistical features of global shape. Finally, we learn a mapping from these features to measurements using regularized linear regression. We perform an extensive evaluation using the CAESAR dataset and demonstrate that the accuracy of our method outperforms state-of-the-art methods.
pdf DOI BibTeX