FaceTalk: Audio-driven Motion Diffusion for Neural Parametric Head Models | Neural Capture and Synthesis – Max Planck Institute for Intelligent Systems

Institute Homepage

Institute Homepage Sign In

Research Overview

Digital Humans in Motion

Digital Multi-Media Forensics

Digitization of Objects and Scenes

Capturing Humans, Objects and Scenes

MICA: Towards Metrical Reconstruction of Human Faces

GAN-Avatar: Controllable Personalized GAN-based Human Head Avatars

INSTA: Instant Volumetric Head Avatars

Motion Synthesis

Imitator: Personalized Speech-driven 3D Facial Animation

FaceTalk: Audio-driven Motion Diffusion for Neural Parametric Head Models

3DiFace: Synthesizing and Editing Holistic 3D Facial Animations

(Generative) Appearance Modelling and Editing

HAAR: Tex-conditioned Generative Model of 3D Strand-based Human Hairstyles

High-Res Facial Appearance Capture from Polarized Smartphone Images

Stable Video Portraits

Neural Capture and Synthesis

FaceTalk: Audio-driven Motion Diffusion for Neural Parametric Head Models

Facetalk — Detailed facial motion generation from audio using neural parametric head models.

FaceTalk is a generative approach designed for synthesizing high-fidelity 3D motion sequences of talking human heads from input audio. To capture the expressive, detailed nature of human heads, including hair, ears, and finer-scale eye movements, we propose to couple speech signal with the latent space of neural parametric head models.