Header logo is


2019


Towards Geometric Understanding of Motion
Towards Geometric Understanding of Motion

Ranjan, A.

University of Tübingen, December 2019 (phdthesis)

Abstract

The motion of the world is inherently dependent on the spatial structure of the world and its geometry. Therefore, classical optical flow methods try to model this geometry to solve for the motion. However, recent deep learning methods take a completely different approach. They try to predict optical flow by learning from labelled data. Although deep networks have shown state-of-the-art performance on classification problems in computer vision, they have not been as effective in solving optical flow. The key reason is that deep learning methods do not explicitly model the structure of the world in a neural network, and instead expect the network to learn about the structure from data. We hypothesize that it is difficult for a network to learn about motion without any constraint on the structure of the world. Therefore, we explore several approaches to explicitly model the geometry of the world and its spatial structure in deep neural networks.

The spatial structure in images can be captured by representing it at multiple scales. To represent multiple scales of images in deep neural nets, we introduce a Spatial Pyramid Network (SpyNet). Such a network can leverage global information for estimating large motions and local information for estimating small motions. We show that SpyNet significantly improves over previous optical flow networks while also being the smallest and fastest neural network for motion estimation. SPyNet achieves a 97% reduction in model parameters over previous methods and is more accurate.

The spatial structure of the world extends to people and their motion. Humans have a very well-defined structure, and this information is useful in estimating optical flow for humans. To leverage this information, we create a synthetic dataset for human optical flow using a statistical human body model and motion capture sequences. We use this dataset to train deep networks and see significant improvement in the ability of the networks to estimate human optical flow.

The structure and geometry of the world affects the motion. Therefore, learning about the structure of the scene together with the motion can benefit both problems. To facilitate this, we introduce Competitive Collaboration, where several neural networks are constrained by geometry and can jointly learn about structure and motion in the scene without any labels. To this end, we show that jointly learning single view depth prediction, camera motion, optical flow and motion segmentation using Competitive Collaboration achieves state-of-the-art results among unsupervised approaches.

Our findings provide support for our hypothesis that explicit constraints on structure and geometry of the world lead to better methods for motion estimation.

ps

PhD Thesis [BibTex]

2019


PhD Thesis [BibTex]


no image
Novel X-ray lenses for direct and coherent imaging

Sanli, U. T.

Universität Stuttgart, Stuttgart, 2019 (phdthesis)

mms

link (url) DOI [BibTex]

link (url) DOI [BibTex]

2018


Model-based Optical Flow: Layers, Learning, and Geometry
Model-based Optical Flow: Layers, Learning, and Geometry

Wulff, J.

Tuebingen University, April 2018 (phdthesis)

Abstract
The estimation of motion in video sequences establishes temporal correspondences between pixels and surfaces and allows reasoning about a scene using multiple frames. Despite being a focus of research for over three decades, computing motion, or optical flow, remains challenging due to a number of difficulties, including the treatment of motion discontinuities and occluded regions, and the integration of information from more than two frames. One reason for these issues is that most optical flow algorithms only reason about the motion of pixels on the image plane, while not taking the image formation pipeline or the 3D structure of the world into account. One approach to address this uses layered models, which represent the occlusion structure of a scene and provide an approximation to the geometry. The goal of this dissertation is to show ways to inject additional knowledge about the scene into layered methods, making them more robust, faster, and more accurate. First, this thesis demonstrates the modeling power of layers using the example of motion blur in videos, which is caused by fast motion relative to the exposure time of the camera. Layers segment the scene into regions that move coherently while preserving their occlusion relationships. The motion of each layer therefore directly determines its motion blur. At the same time, the layered model captures complex blur overlap effects at motion discontinuities. Using layers, we can thus formulate a generative model for blurred video sequences, and use this model to simultaneously deblur a video and compute accurate optical flow for highly dynamic scenes containing motion blur. Next, we consider the representation of the motion within layers. Since, in a layered model, important motion discontinuities are captured by the segmentation into layers, the flow within each layer varies smoothly and can be approximated using a low dimensional subspace. We show how this subspace can be learned from training data using principal component analysis (PCA), and that flow estimation using this subspace is computationally efficient. The combination of the layered model and the low-dimensional subspace gives the best of both worlds, sharp motion discontinuities from the layers and computational efficiency from the subspace. Lastly, we show how layered methods can be dramatically improved using simple semantics. Instead of treating all layers equally, a semantic segmentation divides the scene into its static parts and moving objects. Static parts of the scene constitute a large majority of what is shown in typical video sequences; yet, in such regions optical flow is fully constrained by the depth structure of the scene and the camera motion. After segmenting out moving objects, we consider only static regions, and explicitly reason about the structure of the scene and the camera motion, yielding much better optical flow estimates. Furthermore, computing the structure of the scene allows to better combine information from multiple frames, resulting in high accuracies even in occluded regions. For moving regions, we compute the flow using a generic optical flow method, and combine it with the flow computed for the static regions to obtain a full optical flow field. By combining layered models of the scene with reasoning about the dynamic behavior of the real, three-dimensional world, the methods presented herein push the envelope of optical flow computation in terms of robustness, speed, and accuracy, giving state-of-the-art results on benchmarks and pointing to important future research directions for the estimation of motion in natural scenes.

ps

Official link DOI Project Page [BibTex]


no image
XMCD investigations on new hard magnetic systems

Chen, Y.

Universität Stuttgart, Stuttgart, 2018 (phdthesis)

mms

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
High-Resolution X-ray Ptychography for Magnetic Imaging

Bykova, I.

Universität Stuttgart, Stuttgart, 2018 (phdthesis)

mms

link (url) DOI [BibTex]

link (url) DOI [BibTex]

2014


Modeling the Human Body in 3D: Data Registration and Human Shape Representation
Modeling the Human Body in 3D: Data Registration and Human Shape Representation

Tsoli, A.

Brown University, Department of Computer Science, May 2014 (phdthesis)

ps

pdf [BibTex]

2014


pdf [BibTex]


no image
Schalten der Polarität magnetischer Vortexkerne durch eine Zwei-Frequenzen Anregung und mittels direkter Einkopplung eines Stroms

Sproll, M.

Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), Stuttgart, 2014 (phdthesis)

mms

[BibTex]

[BibTex]


no image
Vortex-Kern-Korrelation in gekoppelten Systemen

Jüllig, P.

Universität Stuttgart, Stuttgart, 2014 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Realization of a new Magnetic Scanning X-ray Microscope and Investigation of Landau Structures under Pulsed Field Excitation

Weigand, M.

Universität Stuttgart, Stuttgart (und Cuvillier Verlag, Göttingen), 2014 (phdthesis)

mms

[BibTex]

[BibTex]


no image
Nanoporous Materials for Hydrogen Storage and H2/D2 Isotope Separation

Oh, H.

Universität Stuttgart, Stuttgart, 2014 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]

2007


no image
On the theory of magnetization dynamics of non-collinear spin systems in the s-d model

De Angeli, L.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

2007


[BibTex]


no image
Zur ab-initio Elektronentheorie des Magnetismus bei endlichen Temperaturen

Dietermann, F.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Röntgenzirkulardichroische Untersuchungen an ferromagnetischen verdünnten Halbleitersystemen

Tietze, T.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Low-dimensional Fe on vicinal Ir(997): Growth and magnetic properties

Kawwam, M.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Micromagnetic simulations of switching processes and the role of thermal fluctuations

Macke, S.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Physisorption von Wasserstoff in neuen Materialien mit gro\sser spezifischer Oberfläche

Schmitz, B.

Universität Bonn, Bonn, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Towards spin injection into silicon

Dash, S. P.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Bestimmung der kritischen Schichtdicken ferromagnetischer Plättchen für Eindomänenverhalten

Soehnle, S.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Zeitaufgelöste Röntgenmikroskopie an magnetischen Mikrostrukturen

Puzic, A.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Vortex dynamics studied by time-resolved X-ray microscopy

Chou, K. W.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Resonante magnetische Reflektometrie an Ferromagnet/Paramagnet Heterostrukturen

Ferreras Paz, V.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Herstellung und Charakterisierung dünner Niob-Schichten auf verschiedenen Substraten

Mayer, M. W. R.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Formation of hard magnetic L10-FePt/FePd monolayers from elemental multilayers

Goo, N. H.

Universität Stuttgart, Stuttgart, 2007 (phdthesis)

mms

link (url) [BibTex]

link (url) [BibTex]


no image
Zur ab-initio Elektronentheorie stark nichtkollinearer Spinsysteme

Köberle, I.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Theorie der Kernspektroskopie mit zirkular polarisierter Gammastrahlung

Engelhart, W.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Untersuchung der Adsorption von Wasserstoff in porösen Materialien

Hönes, K.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]


no image
Untersuchung der mechanischen Eigenschaften dünner Chromschichten

Jüllig, P.

Universität Stuttgart, Stuttgart, 2007 (mastersthesis)

mms

[BibTex]

[BibTex]