Layered image motion with explicit occlusions, temporal consistency, and depth ordering
main paper supplemental material paper and supplemental material in one pdf file
Layered models are a powerful way of describing natural scenes containing smooth surfaces that may overlap and occlude each other. For image motion estimation, such models have a long history but have not achieved the wide use or accuracy of non-layered methods. We present a new probabilistic model of optical flow in layers that addresses many of the shortcomings of previous approaches. In particular, we define a probabilistic graphical model that explicitly captures: 1) occlusions and disocclusions; 2) depth ordering of the layers; 3) temporal consistency of the layer segmentation. Additionally the optical flow in each layer is modeled by a combination of a parametric model and a smooth deviation based on an MRF with a robust spatial prior; the resulting model allows roughness in layers. Finally, a key contribution is the formulation of the layers using an image dependent hidden field prior based on recent models for static scene segmentation. The method achieves state-of-the-art results on the Middlebury benchmark and produces meaningful scene segmentations as well as detected occlusion regions.
| Author(s): | Sun, D. and Sudderth, E. and Black, M. J. |
| Links: | |
| Book Title: | Advances in Neural Information Processing Systems 23 (NIPS) |
| Pages: | 2226--2234 |
| Year: | 2010 |
| Publisher: | MIT Press |
| Project(s): | |
| Bibtex Type: | Conference Paper (inproceedings) |
| Electronic Archiving: | grant_archive |
BibTex
@inproceedings{Sun:NIPS:10,
title = {Layered image motion with explicit occlusions, temporal consistency, and depth ordering},
booktitle = {Advances in Neural Information Processing Systems 23 (NIPS)},
abstract = {Layered models are a powerful way of describing natural scenes containing smooth surfaces that may overlap and occlude each other. For image motion estimation, such models have a long history but have not achieved the wide use or accuracy of non-layered methods. We present a new probabilistic model of optical flow in layers that addresses many of the shortcomings of previous approaches. In particular, we define a probabilistic graphical model that explicitly captures: 1) occlusions and disocclusions; 2) depth ordering of the layers; 3) temporal consistency of the layer segmentation. Additionally the optical flow in each layer is modeled by a combination of a parametric model and a smooth deviation based on an MRF with a robust spatial prior; the resulting model allows roughness in
layers. Finally, a key contribution is the formulation of the layers using an image dependent hidden field prior based on recent models for static scene segmentation. The method achieves state-of-the-art results on the Middlebury benchmark and produces meaningful scene segmentations as well as detected occlusion regions.},
pages = {2226--2234},
publisher = {MIT Press},
year = {2010},
slug = {sun-nips-10},
author = {Sun, D. and Sudderth, E. and Black, M. J.}
}