Animatable humans from monocular RGB(D) videos

We aim to reconstruct animatable humans from monocular RGB(D) videos. Learning user-controlled representation under novel poses remains a challenging problem. To tackle this problem, I will introduce two related methods. First, to reconstruct animatable photo-realistic humans, we integrate observations across frames and encode the appearance at each individual frame by utilizing the human pose and point clouds as the input. In addition, we utilize a temporal transformer to integrate the features of points in the unseen frames and the tracked points in a handful of automatically-selected key frames. Second, to reconstruct the animatable 3D geometry, we propose to represent the human with clothes in the canonical space by learning the implicit functions and utilize the LBS to animate the learned geometry.