Imitator: Personalized Speech-driven 3D Facial Animation | Neural Capture and Synthesis – Max Planck Institute for Intelligent Systems

Institute Homepage

Institute Homepage Sign In

Research Overview

Digital Humans in Motion

Digital Multi-Media Forensics

Digitization of Objects and Scenes

Capturing Humans, Objects and Scenes

MICA: Towards Metrical Reconstruction of Human Faces

GAN-Avatar: Controllable Personalized GAN-based Human Head Avatars

INSTA: Instant Volumetric Head Avatars

Motion Synthesis

Imitator: Personalized Speech-driven 3D Facial Animation

FaceTalk: Audio-driven Motion Diffusion for Neural Parametric Head Models

3DiFace: Synthesizing and Editing Holistic 3D Facial Animations

(Generative) Appearance Modelling and Editing

HAAR: Tex-conditioned Generative Model of 3D Strand-based Human Hairstyles

High-Res Facial Appearance Capture from Polarized Smartphone Images

Stable Video Portraits

Neural Capture and Synthesis

Imitator: Personalized Speech-driven 3D Facial Animation

Imitator — Generating 3D facial animation sequences from audio.

Imitator learns identity-specific details from a short input video of a subject and produces novel facial expressions matching the identity-specific speaking style. A style-agnostic transformer is trained on a large expression dataset which we use as a prior for audio-driven expressions.