Thumb ticker sm headshot2021
Perceiving Systems
Director
no image
Perceiving Systems
Learningflow

Assumptions of brightness constancy and spatial smoothness underlie most optical flow estimation methods. In contrast to standard heuristic formulations, we learn a statistical model of both brightness constancy error and the spatial properties of optical flow using image sequences with associated ground truth flow fields. The result is a complete probabilistic model of optical flow. Specifically, the ground truth enables us to model how the assumption of brightness constancy is violated in naturalistic sequences, resulting in a probabilistic model of "brightness inconstancy". We also generalize previous high-order constancy assumptions, such as gradient constancy, by modeling the constancy of responses to various linear filters in a high-order random field framework. These filters are free variables that can be learned from training data. Additionally we study the spatial structure of the optical flow and how motion boundaries are related to image intensity boundaries. Spatial smoothness is modeled using a Steerable Random Field, where spatial derivatives of the optical flow are steered by the image brightness structure. These models provide a statistical motivation for previous methods and enable the learning of all parameters from training data. All proposed models are quantitatively compared on the Middlebury flow dataset.

Author(s): Sun, Deqing and Roth, Stefan and Lewis, J.P. and Black, Michael J.
Links:
Book Title: European Conf. on Computer Vision, ECCV
Volume: 5304
Pages: 83--97
Year: 2008
Month: October
Series: LNCS
Editors: Forsyth, D. and Torr, P. and Zisserman, A.
Publisher: Springer-Verlag
Bibtex Type: Conference Paper (inproceedings)
DOI: 10.1007/978-3-540-88690-7_7
Electronic Archiving: grant_archive

BibTex

@inproceedings{Sun:ECCV:2008,
  title = {Learning Optical Flow},
  booktitle = {European Conf. on Computer Vision, ECCV},
  abstract = {Assumptions of brightness constancy and spatial smoothness underlie most optical flow estimation methods. In contrast to standard heuristic formulations, we learn a statistical model of both brightness constancy error and the spatial properties of optical flow using image sequences with associated ground truth flow fields. The result is a complete probabilistic model of optical flow. Specifically, the ground truth enables us to model how the assumption of brightness constancy is violated in naturalistic sequences, resulting in a probabilistic model of "brightness inconstancy". We also generalize previous high-order constancy assumptions, such as gradient constancy, by modeling the constancy of responses to various linear filters in a high-order random field framework. These filters are free variables that can be learned from training data. Additionally we study the spatial structure of the optical flow and how motion boundaries are related to image intensity boundaries. Spatial smoothness is modeled using a Steerable Random Field, where spatial derivatives of the optical flow are steered by the image brightness structure. These models provide a statistical motivation for previous methods and enable the learning of all parameters from training data. All proposed models are quantitatively compared on the Middlebury flow dataset.},
  volume = {5304},
  pages = {83--97},
  series = {LNCS},
  editors = {Forsyth, D. and Torr, P. and Zisserman, A.},
  publisher = {Springer-Verlag},
  month = oct,
  year = {2008},
  slug = {sun-eccv-2008},
  author = {Sun, Deqing and Roth, Stefan and Lewis, J.P. and Black, Michael J.},
  month_numeric = {10}
}