Efficient Sparse-to-Dense Optical Flow Estimation using a Learned Basis and Layers
pdf Extended Abstract Supplemental Material Poster Code
We address the elusive goal of estimating optical flow both accurately and efficiently by adopting a sparse-to-dense approach. Given a set of sparse matches, we regress to dense optical flow using a learned set of full-frame basis flow fields. We learn the principal components of natural flow fields using flow computed from four Hollywood movies. Optical flow fields are then compactly approximated as a weighted sum of the basis flow fields. Our new PCA-Flow algorithm robustly estimates these weights from sparse feature matches. The method runs in under 300ms/frame on the MPI-Sintel dataset using a single CPU and is more accurate and significantly faster than popular methods such as LDOF and Classic+NL. The results, however, are too smooth for some applications. Consequently, we develop a novel sparse layered flow method in which each layer is represented by PCA-flow. Unlike existing layered methods, estimation is fast because it uses only sparse matches. We combine information from different layers into a dense flow field using an image-aware MRF. The resulting PCA-Layers method runs in 3.6s/frame, is significantly more accurate than PCA-flow and achieves state-of-the-art performance in occluded regions on MPI-Sintel.
| Author(s): | Jonas Wulff and Michael J. Black |
| Links: | |
| Book Title: | IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2015) |
| Pages: | 120--130 |
| Year: | 2015 |
| Month: | June |
| Project(s): |
|
| BibTeX Type: | Conference Paper (inproceedings) |
| Event Place: | Boston, MA, USA |
| Electronic Archiving: | grant_archive |
BibTeX
@inproceedings{Wulff:CVPR:2015,
title = {Efficient Sparse-to-Dense Optical Flow Estimation using a Learned Basis and Layers},
booktitle = { IEEE Conf. on Computer Vision and Pattern Recognition (CVPR 2015)},
abstract = {We address the elusive goal of estimating optical flow both accurately and efficiently by adopting a sparse-to-dense approach. Given a set of sparse matches, we regress to dense optical flow using a learned set of full-frame basis
flow fields. We learn the principal components of natural flow fields using flow computed from four Hollywood
movies. Optical flow fields are then compactly approximated as a weighted sum of the basis flow fields. Our
new PCA-Flow algorithm robustly estimates these weights from sparse feature matches. The method runs in under
300ms/frame on the MPI-Sintel dataset using a single CPU and is more accurate and significantly faster than popular
methods such as LDOF and Classic+NL. The results, however, are too smooth for some applications. Consequently,
we develop a novel sparse layered flow method in which each layer is represented by PCA-flow. Unlike existing layered
methods, estimation is fast because it uses only sparse matches. We combine information from different layers into
a dense flow field using an image-aware MRF. The resulting PCA-Layers method runs in 3.6s/frame, is significantly
more accurate than PCA-flow and achieves state-of-the-art performance in occluded regions on MPI-Sintel.},
pages = {120--130},
month = jun,
year = {2015},
author = {Wulff, Jonas and Black, Michael J.},
month_numeric = {6}
}