While the accuracy of optical flow estimation has increased markedly, a number of problems remain, most notably the treatment of motion and image boundaries, the tracking of fast but small/thin objects, and the computational complexity of current methods. 

In [], we comprehensively analyze the lessons the field has learned in the past decades. We systematically evaluate techniques to find out what really works. We observe that traditional formulations can achieve competitive performance when implmented using modern practices. Moreover we find that image-mediated spatial smoothing is critical to accuracy and formulate the prevailing ad hoc approach as a principled objective function.

In [] we addrss the problem of small objects that move more than their own spatial extent. While large displacements are usually captured using an image pyramid, this blurs over small objects, making their motion untrackable. To address this we replace images with Distribution Fields, which allow the use of spatial pyramids to capture large motions while preserving high-frequency spatial detail, allowing us to track small objects over large displacements.

In [] we use discrete optimization to estimate optical flow. As naive discretization of the 2D flow space is intractable, we investigate three different strategies, which are able to reduce computation and memory demands by several orders of magnitude. Their combination allows us to estimate large-displacement optical flow both accurately and efficiently, attaining state-of-the-art performance.

PCA-Flow [] takes a non-standard approach to compute optical flow efficiently. Instead of modelling the motion of each pixel as a variable, we treat the full optical flow field as a datapoint in a 500-dimensional subspace, the structure of which is learned from 8 hours of movie data. Computing an optical flow field then becomes equivalent to finding a point in this subspace. This allows us to rapidly compute an approximate, smooth flow field, which can then serve as a building block for other applications, such as layered optical flow. PCA flow is the fastest non-GPU flow method obtaining peformance better than methods like Classic+NL and LDOF.