Header logo is


2018


On the Integration of Optical Flow and Action Recognition
On the Integration of Optical Flow and Action Recognition

Sevilla-Lara, L., Liao, Y., Güney, F., Jampani, V., Geiger, A., Black, M. J.

In German Conference on Pattern Recognition (GCPR), LNCS 11269, pages: 281-297, Springer, Cham, October 2018 (inproceedings)

Abstract
Most of the top performing action recognition methods use optical flow as a "black box" input. Here we take a deeper look at the combination of flow and action recognition, and investigate why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better. In particular, we investigate the impact of different flow algorithms and input transformations to better understand how these affect a state-of-the-art action recognition method. Furthermore, we fine tune two neural-network flow methods end-to-end on the most widely used action recognition dataset (UCF101). Based on these experiments, we make the following five observations: 1) optical flow is useful for action recognition because it is invariant to appearance, 2) optical flow methods are optimized to minimize end-point-error (EPE), but the EPE of current methods is not well correlated with action recognition performance, 3) for the flow methods tested, accuracy at boundaries and at small displacements is most correlated with action recognition performance, 4) training optical flow to minimize classification error instead of minimizing EPE improves recognition performance, and 5) optical flow learned for the task of action recognition differs from traditional optical flow especially inside the human body and at the boundary of the body. These observations may encourage optical flow researchers to look beyond EPE as a goal and guide action recognition researchers to seek better motion cues, leading to a tighter integration of the optical flow and action recognition communities.

avg ps

arXiv DOI [BibTex]

2018


arXiv DOI [BibTex]


Towards Robust Visual Odometry with a Multi-Camera System
Towards Robust Visual Odometry with a Multi-Camera System

Liu, P., Geppert, M., Heng, L., Sattler, T., Geiger, A., Pollefeys, M.

In International Conference on Intelligent Robots and Systems (IROS) 2018, International Conference on Intelligent Robots and Systems, October 2018 (inproceedings)

Abstract
We present a visual odometry (VO) algorithm for a multi-camera system and robust operation in challenging environments. Our algorithm consists of a pose tracker and a local mapper. The tracker estimates the current pose by minimizing photometric errors between the most recent keyframe and the current frame. The mapper initializes the depths of all sampled feature points using plane-sweeping stereo. To reduce pose drift, a sliding window optimizer is used to refine poses and structure jointly. Our formulation is flexible enough to support an arbitrary number of stereo cameras. We evaluate our algorithm thoroughly on five datasets. The datasets were captured in different conditions: daytime, night-time with near-infrared (NIR) illumination and night-time without NIR illumination. Experimental results show that a multi-camera setup makes the VO more robust to challenging environments, especially night-time conditions, in which a single stereo configuration fails easily due to the lack of features.

avg

pdf Project Page [BibTex]

pdf Project Page [BibTex]


Learning Priors for Semantic 3D Reconstruction
Learning Priors for Semantic 3D Reconstruction

Cherabier, I., Schönberger, J., Oswald, M., Pollefeys, M., Geiger, A.

In Computer Vision – ECCV 2018, Springer International Publishing, Cham, September 2018 (inproceedings)

Abstract
We present a novel semantic 3D reconstruction framework which embeds variational regularization into a neural network. Our network performs a fixed number of unrolled multi-scale optimization iterations with shared interaction weights. In contrast to existing variational methods for semantic 3D reconstruction, our model is end-to-end trainable and captures more complex dependencies between the semantic labels and the 3D geometry. Compared to previous learning-based approaches to 3D reconstruction, we integrate powerful long-range dependencies using variational coarse-to-fine optimization. As a result, our network architecture requires only a moderate number of parameters while keeping a high level of expressiveness which enables learning from very little data. Experiments on real and synthetic datasets demonstrate that our network achieves higher accuracy compared to a purely variational approach while at the same time requiring two orders of magnitude less iterations to converge. Moreover, our approach handles ten times more semantic class labels using the same computational resources.

avg

pdf suppmat Project Page Video DOI Project Page [BibTex]

pdf suppmat Project Page Video DOI Project Page [BibTex]


no image
Discovering and Teaching Optimal Planning Strategies

Lieder, F., Callaway, F., Krueger, P. M., Das, P., Griffiths, T. L., Gul, S.

In The 14th biannual conference of the German Society for Cognitive Science, GK, September 2018, Falk Lieder and Frederick Callaway contributed equally to this publication. (inproceedings)

Abstract
How should we think and decide, and how can we learn to make better decisions? To address these questions we formalize the discovery of cognitive strategies as a metacognitive reinforcement learning problem. This formulation leads to a computational method for deriving optimal cognitive strategies and a feedback mechanism for accelerating the process by which people learn how to make better decisions. As a proof of concept, we apply our approach to develop an intelligent system that teaches people optimal planning stratgies. Our training program combines a novel process-tracing paradigm that makes peoples latent planning strategies observable with an intelligent system that gives people feedback on how their planning strategy could be improved. The pedagogy of our intelligent tutor is based on the theory that people discover their cognitive strategies through metacognitive reinforcement learning. Concretely, the tutor’s feedback is designed to maximally accelerate people’s metacognitive reinforcement learning towards the optimal cognitive strategy. A series of four experiments confirmed that training with the cognitive tutor significantly improved people’s decision-making competency: Experiment 1 demonstrated that the cognitive tutor’s feedback accelerates participants’ metacognitive learning. Experiment 2 found that this training effect transfers to more difficult planning problems in more complex environments. Experiment 3 found that these transfer effects are retained for at least 24 hours after the training. Finally, Experiment 4 found that practicing with the cognitive tutor conveys additional benefits above and beyond verbal description of the optimal planning strategy. The results suggest that promoting metacognitive reinforcement learning with optimal feedback is a promising approach to improving the human mind.

re

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


Unsupervised Learning of Multi-Frame Optical Flow with Occlusions
Unsupervised Learning of Multi-Frame Optical Flow with Occlusions

Janai, J., Güney, F., Ranjan, A., Black, M. J., Geiger, A.

In European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, vol 11220, pages: 713-731, Springer, Cham, September 2018 (inproceedings)

avg ps

pdf suppmat Video Project Page DOI Project Page [BibTex]

pdf suppmat Video Project Page DOI Project Page [BibTex]


no image
Discovering Rational Heuristics for Risky Choice

Gul, S., Krueger, P. M., Callaway, F., Griffiths, T. L., Lieder, F.

The 14th biannual conference of the German Society for Cognitive Science, GK, The 14th biannual conference of the German Society for Cognitive Science, GK, September 2018 (conference)

Abstract
How should we think and decide to make the best possible use of our precious time and limited cognitive resources? And how do people’s cognitive strategies compare to this ideal? We study these questions in the domain of multi-alternative risky choice using the methodology of resource-rational analysis. To answer the first question, we leverage a new meta-level reinforcement learning algorithm to derive optimal heuristics for four different risky choice environments. We find that our method rediscovers two fast-and-frugal heuristics that people are known to use, namely Take-The-Best and choosing randomly, as resource-rational strategies for specific environments. Our method also discovered a novel heuristic that combines elements of Take-The-Best and Satisficing. To answer the second question, we use the Mouselab paradigm to measure how people’s decision strategies compare to the predictions of our resource-rational analysis. We found that our resource-rational analysis correctly predicted which strategies people use and under which conditions they use them. While people generally tend to make rational use of their limited resources overall, their strategy choices do not always fully exploit the structure of each decision problem. Overall, people’s decision operations were about 88% as resource-rational as they could possibly be. A formal model comparison confirmed that our resource-rational model explained people’s decision strategies significantly better than the Directed Cognition model of Gabaix et al. (2006). Our study is a proof-of-concept that optimal cognitive strategies can be automatically derived from the principle of resource-rationality. Our results suggest that resource-rational analysis is a promising approach for uncovering people’s cognitive strategies and revisiting the debate about human rationality with a more realistic normative standard.

re

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images
SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images

Coors, B., Condurache, A. P., Geiger, A.

European Conference on Computer Vision (ECCV), September 2018 (conference)

Abstract
Omnidirectional cameras offer great benefits over classical cameras wherever a wide field of view is essential, such as in virtual reality applications or in autonomous robots. Unfortunately, standard convolutional neural networks are not well suited for this scenario as the natural projection surface is a sphere which cannot be unwrapped to a plane without introducing significant distortions, particularly in the polar regions. In this work, we present SphereNet, a novel deep learning framework which encodes invariance against such distortions explicitly into convolutional neural networks. Towards this goal, SphereNet adapts the sampling locations of the convolutional filters, effectively reversing distortions, and wraps the filters around the sphere. By building on regular convolutions, SphereNet enables the transfer of existing perspective convolutional neural network models to the omnidirectional case. We demonstrate the effectiveness of our method on the tasks of image classification and object detection, exploiting two newly created semi-synthetic and real-world omnidirectional datasets.

avg

pdf suppmat Project Page [BibTex]


no image
Learning to Select Computations

Callaway, F., Gul, S., Krueger, P. M., Griffiths, T. L., Lieder, F.

In Uncertainty in Artificial Intelligence: Proceedings of the Thirty-Fourth Conference, August 2018, Frederick Callaway and Sayan Gul and Falk Lieder contributed equally to this publication. (inproceedings)

Abstract
The efficient use of limited computational resources is an essential ingredient of intelligence. Selecting computations optimally according to rational metareasoning would achieve this, but this is computationally intractable. Inspired by psychology and neuroscience, we propose the first concrete and domain-general learning algorithm for approximating the optimal selection of computations: Bayesian metalevel policy search (BMPS). We derive this general, sample-efficient search algorithm for a computation-selecting metalevel policy based on the insight that the value of information lies between the myopic value of information and the value of perfect information. We evaluate BMPS on three increasingly difficult metareasoning problems: when to terminate computation, how to allocate computation between competing options, and planning. Across all three domains, BMPS achieved near-optimal performance and compared favorably to previously proposed metareasoning heuristics. Finally, we demonstrate the practical utility of BMPS in an emergency management scenario, even accounting for the overhead of metareasoning.

re

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


no image
Assessment Of Atypical Motor Development In Infants Through Toy-Stimulated Play And Center Of Pressure Analysis

Zhao, S., Mohan, M., Torres, W. O., Bogen, D. K., Shofer, F. S., Prosser, L., Loeb, H., Johnson, M. J.

In Proceedings of the Annual Rehabilitation Engineering and Assistive Technology Society of North America (RESNA) Conference, Arlington, USA, July 2018 (inproceedings)

Abstract
There is a need to identify measures and create systems to assess motor development at an early stage. Center of Pressure (CoP) is a quantifiable metric that has been used to investigate postural control in healthy young children [6], children with CP [7], and infants just beginning to sit [8]. It was found that infants born prematurely exhibit different patterns of CoP movement than infants born full-term when assessing development impairments relating to postural control [9]. Preterm infants exhibited greater CoP excursions but had greater variability in their movements than fullterm infants. Our solution, the Play And Neuro-Development Assessment (PANDA) Gym, is a sensorized environment that aims to provide early diagnosis of neuromotor disorder in infants and improve current screening processes by providing quantitative measures rather than subjective ones, and promoting natural play with the stimulus of toys. Previous studies have documented stages in motor development in infants [10, 11], and developmental delays could become more apparent through toy interactions. This study examines the sensitivity of the pressure-sensitive mat subsystem to detect differences in CoP movement patterns for preterm and fullterm infants less than 6 months of age, with varying risk levels. This study aims to distinguish between typical and atypical motor development through assessment of the CoP data of infants in a natural play environment, in conditions where movement may be further stimulated with the presence of a toy.

hi

link (url) [BibTex]

link (url) [BibTex]


Robust Dense Mapping for Large-Scale Dynamic Environments
Robust Dense Mapping for Large-Scale Dynamic Environments

Barsan, I. A., Liu, P., Pollefeys, M., Geiger, A.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2018, IEEE, International Conference on Robotics and Automation, May 2018 (inproceedings)

Abstract
We present a stereo-based dense mapping algorithm for large-scale dynamic urban environments. In contrast to other existing methods, we simultaneously reconstruct the static background, the moving objects, and the potentially moving but currently stationary objects separately, which is desirable for high-level mobile robotic tasks such as path planning in crowded environments. We use both instance-aware semantic segmentation and sparse scene flow to classify objects as either background, moving, or potentially moving, thereby ensuring that the system is able to model objects with the potential to transition from static to dynamic, such as parked cars. Given camera poses estimated from visual odometry, both the background and the (potentially) moving objects are reconstructed separately by fusing the depth maps computed from the stereo input. In addition to visual odometry, sparse scene flow is also used to estimate the 3D motions of the detected moving objects, in order to reconstruct them accurately. A map pruning technique is further developed to improve reconstruction accuracy and reduce memory consumption, leading to increased scalability. We evaluate our system thoroughly on the well-known KITTI dataset. Our system is capable of running on a PC at approximately 2.5Hz, with the primary bottleneck being the instance-aware semantic segmentation, which is a limitation we hope to address in future work.

avg

pdf Video Project Page Project Page [BibTex]

pdf Video Project Page Project Page [BibTex]


A resource-rational analysis of human planning
A resource-rational analysis of human planning

Callaway, F., Lieder, F., Das, P., Gul, S., Krueger, P. M., Griffiths, T. L.

In Proceedings of the 40th Annual Conference of the Cognitive Science Society, May 2018, Frederick Callaway and Falk Lieder contributed equally to this publication. (inproceedings)

Abstract
People's cognitive strategies are jointly shaped by function and computational constraints. Resource-rational analysis leverages these constraints to derive rational models of people's cognitive strategies from the assumption that people make rational use of limited cognitive resources. We present a resource-rational analysis of planning and evaluate its predictions in a newly developed process tracing paradigm. In Experiment 1, we find that a resource-rational planning strategy predicts the process by which people plan more accurately than previous models of planning. Furthermore, in Experiment 2, we find that it also captures how people's planning strategies adapt to the structure of the environment. In addition, our approach allows us to quantify for the first time how close people's planning strategies are to being resource-rational and to characterize in which ways they conform to and deviate from optimal planning.

re

DOI [BibTex]

DOI [BibTex]


RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials

Paschalidou, D., Ulusoy, A. O., Schmitt, C., Gool, L., Geiger, A.

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2018 (inproceedings)

Abstract
In this paper, we consider the problem of reconstructing a dense 3D model using images captured from different views. Recent methods based on convolutional neural networks (CNN) allow learning the entire task from data. However, they do not incorporate the physics of image formation such as perspective geometry and occlusion. Instead, classical approaches based on Markov Random Fields (MRF) with ray-potentials explicitly model these physical processes, but they cannot cope with large surface appearance variations across different viewpoints. In this paper, we propose RayNet, which combines the strengths of both frameworks. RayNet integrates a CNN that learns view-invariant feature representations with an MRF that explicitly encodes the physics of perspective projection and occlusion. We train RayNet end-to-end using empirical risk minimization. We thoroughly evaluate our approach on challenging real-world datasets and demonstrate its benefits over a piece-wise trained baseline, hand-crafted models as well as other learning-based approaches.

avg

pdf suppmat Video Project Page code Poster Project Page [BibTex]

pdf suppmat Video Project Page code Poster Project Page [BibTex]


no image
Travelling Ultrasonic Wave Enhances Keyclick Sensation

Gueorguiev, D., Kaci, A., Amberg, M., Giraud, F., Lemaire-Semail, B.

In Haptics: Science, Technology, and Applications, pages: 302-312, Springer International Publishing, Cham, 2018 (inproceedings)

Abstract
A realistic keyclick sensation is a serious challenge for haptic feedback since vibrotactile rendering faces the limitation of the absence of contact force as experienced on physical buttons. It has been shown that creating a keyclick sensation is possible with stepwise ultrasonic friction modulation. However, the intensity of the sensation is limited by the impedance of the fingertip and by the absence of a lateral force component external to the finger. In our study, we compare this technique to rendering with an ultrasonic travelling wave, which exerts a lateral force on the fingertip. For both techniques, participants were asked to report the detection (or not) of a keyclick during a forced choice one interval procedure. In experiment 1, participants could press the surface as many time as they wanted for a given trial. In experiment 2, they were constrained to press only once. The results show a lower perceptual threshold for travelling waves. Moreover, participants pressed less times per trial and exerted smaller normal force on the surface. The subjective quality of the sensation was found similar for both techniques. In general, haptic feedback based on travelling ultrasonic waves is promising for applications without lateral motion of the finger.

hi

[BibTex]

[BibTex]


no image
Enhanced Non-Steady Gliding Performance of the MultiMo-Bat through Optimal Airfoil Configuration and Control Strategy

Kim, H., Woodward, M. A., Sitti, M.

In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 1382-1388, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


Deep Marching Cubes: Learning Explicit Surface Representations
Deep Marching Cubes: Learning Explicit Surface Representations

Liao, Y., Donne, S., Geiger, A.

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2018 (inproceedings)

Abstract
Existing learning based solutions to 3D surface prediction cannot be trained end-to-end as they operate on intermediate representations (eg, TSDF) from which 3D surface meshes must be extracted in a post-processing step (eg, via the marching cubes algorithm). In this paper, we investigate the problem of end-to-end 3D surface prediction. We first demonstrate that the marching cubes algorithm is not differentiable and propose an alternative differentiable formulation which we insert as a final layer into a 3D convolutional neural network. We further propose a set of loss functions which allow for training our model with sparse point supervision. Our experiments demonstrate that the model allows for predicting sub-voxel accurate 3D shapes of arbitrary topology. Additionally, it learns to complete shapes and to separate an object's inside from its outside even in the presence of sparse and incomplete ground truth. We investigate the benefits of our approach on the task of inferring shapes from 3D point clouds. Our model is flexible and can be combined with a variety of shape encoder and shape inference techniques.

avg

pdf suppmat Video Project Page Poster Project Page [BibTex]

pdf suppmat Video Project Page Poster Project Page [BibTex]


Semantic Visual Localization
Semantic Visual Localization

Schönberger, J., Pollefeys, M., Geiger, A., Sattler, T.

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2018 (inproceedings)

Abstract
Robust visual localization under a wide range of viewing conditions is a fundamental problem in computer vision. Handling the difficult cases of this problem is not only very challenging but also of high practical relevance, eg, in the context of life-long localization for augmented reality or autonomous robots. In this paper, we propose a novel approach based on a joint 3D geometric and semantic understanding of the world, enabling it to succeed under conditions where previous approaches failed. Our method leverages a novel generative model for descriptor learning, trained on semantic scene completion as an auxiliary task. The resulting 3D descriptors are robust to missing observations by encoding high-level 3D geometric and semantic information. Experiments on several challenging large-scale localization datasets demonstrate reliable localization under extreme viewpoint, illumination, and geometry changes.

avg

pdf suppmat Poster Project Page [BibTex]

pdf suppmat Poster Project Page [BibTex]


no image
Exploring Fingers’ Limitation of Texture Density Perception on Ultrasonic Haptic Displays

Kalantari, F., Gueorguiev, D., Lank, E., Bremard, N., Grisoni, L.

In Haptics: Science, Technology, and Applications, pages: 354-365, Springer International Publishing, Cham, 2018 (inproceedings)

Abstract
Recent research in haptic feedback is motivated by the crucial role that tactile perception plays in everyday touch interactions. In this paper, we describe psychophysical experiments to investigate the perceptual threshold of individual fingers on both the right and left hand of right-handed participants using active dynamic touch for spatial period discrimination of both sinusoidal and square-wave gratings on ultrasonic haptic touchscreens. Both one-finger and multi-finger touch were studied and compared. Our results indicate that users' finger identity (index finger, middle finger, etc.) significantly affect the perception of both gratings in the case of one-finger exploration. We show that index finger and thumb are the most sensitive in all conditions whereas little finger followed by ring are the least sensitive for haptic perception. For multi-finger exploration, the right hand was found to be more sensitive than the left hand for both gratings. Our findings also demonstrate similar perception sensitivity between multi-finger exploration and the index finger of users' right hands (i.e. dominant hand in our study), while significant difference was found between single and multi-finger perception sensitivity for the left hand.

hi

[BibTex]

[BibTex]


Which Training Methods for GANs do actually Converge?
Which Training Methods for GANs do actually Converge?

Mescheder, L., Geiger, A., Nowozin, S.

International Conference on Machine learning (ICML), 2018 (conference)

Abstract
Recent work has shown local convergence of GAN training for absolutely continuous data and generator distributions. In this paper, we show that the requirement of absolute continuity is necessary: we describe a simple yet prototypical counterexample showing that in the more realistic case of distributions that are not absolutely continuous, unregularized GAN training is not always convergent. Furthermore, we discuss regularization strategies that were recently proposed to stabilize GAN training. Our analysis shows that GAN training with instance noise or zero-centered gradient penalties converges. On the other hand, we show that Wasserstein-GANs and WGAN-GP with a finite number of discriminator updates per generator update do not always converge to the equilibrium point. We discuss these results, leading us to a new explanation for the stability problems of GAN training. Based on our analysis, we extend our convergence results to more general GANs and prove local convergence for simplified gradient penalties even if the generator and data distributions lie on lower dimensional manifolds. We find these penalties to work well in practice and use them to learn high-resolution generative image models for a variety of datasets with little hyperparameter tuning.

avg

code video paper supplement slides poster Project Page [BibTex]


no image
Collectives of Spinning Mobile Microrobots for Navigation and Object Manipulation at the Air-Water Interface

Wang, W., Kishore, V., Koens, L., Lauga, E., Sitti, M.

In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 1-9, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


Learning 3D Shape Completion from Laser Scan Data with Weak Supervision
Learning 3D Shape Completion from Laser Scan Data with Weak Supervision

Stutz, D., Geiger, A.

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2018 (inproceedings)

Abstract
3D shape completion from partial point clouds is a fundamental problem in computer vision and computer graphics. Recent approaches can be characterized as either data-driven or learning-based. Data-driven approaches rely on a shape model whose parameters are optimized to fit the observations. Learning-based approaches, in contrast, avoid the expensive optimization step and instead directly predict the complete shape from the incomplete observations using deep neural networks. However, full supervision is required which is often not available in practice. In this work, we propose a weakly-supervised learning-based approach to 3D shape completion which neither requires slow optimization nor direct supervision. While we also learn a shape prior on synthetic data, we amortize, ie, learn, maximum likelihood fitting using deep neural networks resulting in efficient shape completion without sacrificing accuracy. Tackling 3D shape completion of cars on ShapeNet and KITTI, we demonstrate that the proposed amortized maximum likelihood approach is able to compete with a fully supervised baseline and a state-of-the-art data-driven approach while being significantly faster. On ModelNet, we additionally show that the approach is able to generalize to other object categories as well.

avg

pdf suppmat Project Page Poster Project Page [BibTex]

pdf suppmat Project Page Poster Project Page [BibTex]


no image
Endo-VMFuseNet: A Deep Visual-Magnetic Sensor Fusion Approach for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Gilbert, H. B., Sari, A. E., Soylu, U., Sitti, M.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 1-7, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Endosensorfusion: Particle filtering-based multi-sensory data fusion with switching state-space model for endoscopic capsule robots

Turan, M., Almalioglu, Y., Gilbert, H., Araujo, H., Cemgil, T., Sitti, M.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 1-8, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


Learning Transformation Invariant Representations with Weak Supervision
Learning Transformation Invariant Representations with Weak Supervision

Coors, B., Condurache, A., Mertins, A., Geiger, A.

In International Conference on Computer Vision Theory and Applications, International Conference on Computer Vision Theory and Applications, 2018 (inproceedings)

Abstract
Deep convolutional neural networks are the current state-of-the-art solution to many computer vision tasks. However, their ability to handle large global and local image transformations is limited. Consequently, extensive data augmentation is often utilized to incorporate prior knowledge about desired invariances to geometric transformations such as rotations or scale changes. In this work, we combine data augmentation with an unsupervised loss which enforces similarity between the predictions of augmented copies of an input sample. Our loss acts as an effective regularizer which facilitates the learning of transformation invariant representations. We investigate the effectiveness of the proposed similarity loss on rotated MNIST and the German Traffic Sign Recognition Benchmark (GTSRB) in the context of different classification models including ladder networks. Our experiments demonstrate improvements with respect to the standard data augmentation approach for supervised and semi-supervised learning tasks, in particular in the presence of little annotated data. In addition, we analyze the performance of the proposed approach with respect to its hyperparameters, including the strength of the regularization as well as the layer where representation similarity is enforced.

avg

pdf [BibTex]

pdf [BibTex]

2015


Exploiting Object Similarity in 3D Reconstruction
Exploiting Object Similarity in 3D Reconstruction

Zhou, C., Güney, F., Wang, Y., Geiger, A.

In International Conference on Computer Vision (ICCV), December 2015 (inproceedings)

Abstract
Despite recent progress, reconstructing outdoor scenes in 3D from movable platforms remains a highly difficult endeavor. Challenges include low frame rates, occlusions, large distortions and difficult lighting conditions. In this paper, we leverage the fact that the larger the reconstructed area, the more likely objects of similar type and shape will occur in the scene. This is particularly true for outdoor scenes where buildings and vehicles often suffer from missing texture or reflections, but share similarity in 3D shape. We take advantage of this shape similarity by locating objects using detectors and jointly reconstructing them while learning a volumetric model of their shape. This allows us to reduce noise while completing missing surfaces as objects of similar shape benefit from all observations for the respective category. We evaluate our approach with respect to LIDAR ground truth on a novel challenging suburban dataset and show its advantages over the state-of-the-art.

avg ps

pdf suppmat [BibTex]

2015


pdf suppmat [BibTex]


FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation
FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation

Lenz, P., Geiger, A., Urtasun, R.

In International Conference on Computer Vision (ICCV), International Conference on Computer Vision (ICCV), December 2015 (inproceedings)

Abstract
One of the most popular approaches to multi-target tracking is tracking-by-detection. Current min-cost flow algorithms which solve the data association problem optimally have three main drawbacks: they are computationally expensive, they assume that the whole video is given as a batch, and they scale badly in memory and computation with the length of the video sequence. In this paper, we address each of these issues, resulting in a computationally and memory-bounded solution. First, we introduce a dynamic version of the successive shortest-path algorithm which solves the data association problem optimally while reusing computation, resulting in faster inference than standard solvers. Second, we address the optimal solution to the data association problem when dealing with an incoming stream of data (i.e., online setting). Finally, we present our main contribution which is an approximate online solution with bounded memory and computation which is capable of handling videos of arbitrary length while performing tracking in real time. We demonstrate the effectiveness of our algorithms on the KITTI and PETS2009 benchmarks and show state-of-the-art performance, while being significantly faster than existing solvers.

avg ps

pdf suppmat video project [BibTex]

pdf suppmat video project [BibTex]


Towards Probabilistic Volumetric Reconstruction using Ray Potentials
Towards Probabilistic Volumetric Reconstruction using Ray Potentials

(Best Paper Award)

Ulusoy, A. O., Geiger, A., Black, M. J.

In 3D Vision (3DV), 2015 3rd International Conference on, pages: 10-18, Lyon, October 2015 (inproceedings)

Abstract
This paper presents a novel probabilistic foundation for volumetric 3-d reconstruction. We formulate the problem as inference in a Markov random field, which accurately captures the dependencies between the occupancy and appearance of each voxel, given all input images. Our main contribution is an approximate highly parallelized discrete-continuous inference algorithm to compute the marginal distributions of each voxel's occupancy and appearance. In contrast to the MAP solution, marginals encode the underlying uncertainty and ambiguity in the reconstruction. Moreover, the proposed algorithm allows for a Bayes optimal prediction with respect to a natural reconstruction loss. We compare our method to two state-of-the-art volumetric reconstruction algorithms on three challenging aerial datasets with LIDAR ground truth. Our experiments demonstrate that the proposed algorithm compares favorably in terms of reconstruction accuracy and the ability to expose reconstruction uncertainty.

avg ps

code YouTube pdf suppmat DOI Project Page [BibTex]

code YouTube pdf suppmat DOI Project Page [BibTex]


Compliant wing design for a flapping wing micro air vehicle
Compliant wing design for a flapping wing micro air vehicle

Colmenares, D., Kania, R., Zhang, W., Sitti, M.

In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, pages: 32-39, September 2015 (inproceedings)

Abstract
In this work, we examine several wing designs for a motor-driven, flapping-wing micro air vehicle capable of liftoff. The full system consists of two wings independently driven by geared pager motors that include a spring in parallel with the output shaft. The linear transmission allows for resonant operation, while control is achieved by direct drive of the wing angle. Wings used in previous work were chosen to be fully rigid for simplicity of modeling and fabrication. However, biological wings are highly flexible and other micro air vehicles have successfully utilized flexible wing structures for specialized tasks. The goal of our study is to determine if wing flexibility can be generally used to increase wing performance. Two approaches to lift improvement using flexible wings are explored, resonance of the wing cantilever structure and dynamic wing twisting. We design and test several wings that are compared using different figures of merit. A twisted design improved lift per power by 73.6% and maximum lift production by 53.2% compared to the original rigid design. Wing twist is then modeled in order to propose optimal wing twist profiles that can maximize either wing efficiency or lift production.

pi

DOI [BibTex]

DOI [BibTex]


no image
Millimeter-scale magnetic swimmers using elastomeric undulations

Zhang, J., Diller, E.

In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 1706-1711, September 2015 (inproceedings)

Abstract
This paper presents a new soft-bodied millimeterscale swimmer actuated by rotating uniform magnetic fields. The proposed swimmer moves through internal undulatory deformations, resulting from a magnetization profile programmed into its body. To understand the motion of the swimmer, a mathematical model is developed to describe the general relationship between the deflection of a flexible strip and its magnetization profile. As a special case, the situation of the swimmer on the water surface is analyzed and predictions made by the model are experimentally verified. Experimental results show the controllability of the proposed swimmer under a computer vision-based closed-loop controller. The swimmers have nominal dimensions of 1.5×4.9×0.06 mm and a top speed of 50 mm/s (10 body lengths per second). Waypoint following and multiagent control are demonstrated for swimmers constrained at the air-water interface and underwater swimming is also shown, suggesting the promising potential of this type of swimmer in biomedical and microfluidic applications.

pi

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Assessing human-human therapy kinematics for retargeting to human-robot therapy

Johnson, M. J., Christopher, S. M., Mohan, M., Mendonca, R.

In Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR), Singapore, August 2015 (inproceedings)

Abstract
In this paper, we present experiments examining the accuracy of data collected from a Kinect sensor for capturing close interactive actions of a therapist with a patient during stroke rehabilitation. Our long-term goal is to map human-human interactions such as these patient-therapist ones onto human-robot interactions. In many robot interaction scenarios, the robot does not mimic interaction between two or more humans, which is a major part of stroke therapy. The Kinect works for functional tasks such as a reaching task where the interaction to be retargeted by the robot is minimal to none; though this data is not good for a functional task involving touching another person. We demonstrate that the noisy data from Kinect does not produce a system robust enough to be for remapping to a humanoid robot a therapit's movements when in contact with a person.

hi

DOI [BibTex]

DOI [BibTex]


Displets: Resolving Stereo Ambiguities using Object Knowledge
Displets: Resolving Stereo Ambiguities using Object Knowledge

Güney, F., Geiger, A.

In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2015, pages: 4165-4175, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2015 (inproceedings)

Abstract
Stereo techniques have witnessed tremendous progress over the last decades, yet some aspects of the problem still remain challenging today. Striking examples are reflecting and textureless surfaces which cannot easily be recovered using traditional local regularizers. In this paper, we therefore propose to regularize over larger distances using object-category specific disparity proposals (displets) which we sample using inverse graphics techniques based on a sparse disparity estimate and a semantic segmentation of the image. The proposed displets encode the fact that objects of certain categories are not arbitrarily shaped but typically exhibit regular structures. We integrate them as non-local regularizer for the challenging object class 'car' into a superpixel based CRF framework and demonstrate its benefits on the KITTI stereo evaluation.

avg ps

pdf abstract suppmat [BibTex]

pdf abstract suppmat [BibTex]


no image
Toward a large-scale visuo-haptic dataset for robotic learning

Burka, A., Hu, S., Krishnan, S., Kuchenbecker, K. J., Hendricks, L. A., Gao, Y., Darrell, T.

In Proc. CVPR Workshop on the Future of Datasets in Vision, 2015 (inproceedings)

hi

Project Page [BibTex]

Project Page [BibTex]


Object Scene Flow for Autonomous Vehicles
Object Scene Flow for Autonomous Vehicles

Menze, M., Geiger, A.

In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2015, pages: 3061-3070, IEEE, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June 2015 (inproceedings)

Abstract
This paper proposes a novel model and dataset for 3D scene flow estimation with an application to autonomous driving. Taking advantage of the fact that outdoor scenes often decompose into a small number of independently moving objects, we represent each element in the scene by its rigid motion parameters and each superpixel by a 3D plane as well as an index to the corresponding object. This minimal representation increases robustness and leads to a discrete-continuous CRF where the data term decomposes into pairwise potentials between superpixels and objects. Moreover, our model intrinsically segments the scene into its constituting dynamic components. We demonstrate the performance of our model on existing benchmarks as well as a novel realistic dataset with scene flow ground truth. We obtain this dataset by annotating 400 dynamic scenes from the KITTI raw data collection using detailed 3D CAD models for all vehicles in motion. Our experiments also reveal novel challenges which can't be handled by existing methods.

avg ps

pdf abstract suppmat DOI [BibTex]

pdf abstract suppmat DOI [BibTex]


no image
Detecting Lumps in Simulated Tissue via Palpation with a BioTac

Hui, J., Block, A., Kuchenbecker, K. J.

In Proc. IEEE World Haptics Conference, 2015, Work-in-progress paper. Poster presentation given by Hui (inproceedings)

hi

[BibTex]

[BibTex]


no image
Analysis of the Instrument Vibrations and Contact Forces Caused by an Expert Robotic Surgeon Doing FRS Tasks

Brown, J. D., O’Brien, C., Miyasaka, K., Dumon, K. R., Kuchenbecker, K. J.

In Proc. Hamlyn Symposium on Medical Robotics, pages: 75-76, London, England, June 2015, Poster presentation given by Brown (inproceedings)

hi

[BibTex]

[BibTex]


no image
Should Haptic Texture Vibrations Respond to User Force and Speed?

Culbertson, H., Kuchenbecker, K. J.

In IEEE World Haptics Conference, pages: 106 - 112, Evanston, Illinois, USA, June 2015, Oral presentation given by Culbertson (inproceedings)

hi

[BibTex]

[BibTex]


no image
Enabling the Baxter Robot to Play Hand-Clapping Games

Fitter, N. T., Neuburger, M., Kuchenbecker, K. J.

In Proc. IEEE World Haptics Conference, June 2015, Work-in-progress paper. Poster presentation given by Fitter (inproceedings)

hi

[BibTex]

[BibTex]


Fiberbot: A miniature crawling robot using a directional fibrillar pad
Fiberbot: A miniature crawling robot using a directional fibrillar pad

Han, Y., Marvi, H., Sitti, M.

In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pages: 3122-3127, May 2015 (inproceedings)

Abstract
Vibration-driven locomotion has been widely used for crawling robot studies. Such robots usually have a vibration motor as the actuator and a fibrillar structure for providing directional friction on the substrate. However, there has not been any studies about the effect of fiber structure on robot crawling performance. In this paper, we develop Fiberbot, a custom made mini vibration robot, for studying the effect of fiber angle on robot velocity, steering, and climbing performance. It is known that the friction force with and against fibers depends on the fiber angle. Thus, we first present a new fabrication method for making millimeter scale fibers at a wide range of angles. We then show that using 30° angle fibers that have the highest friction anisotropy (ratio of backward to forward friction force) among the other fibers we fabricated in this study, Fiberbot speed on glass increases to 13.8±0.4 cm/s (compared to ν = 0.6±0.1 cm/s using vertical fibers). We also demonstrate that the locomotion direction of Fiberbot depends on the tilting direction of fibers and we can steer the robot by rotating the fiber pad. Fiberbot could also climb on glass at inclinations of up to 10° when equipped with fibers of high friction anisotropy. We show that adding a rigid tail to the robot it can climb on glass at 25° inclines. Moreover, the robot is able to crawl on rough surfaces such as wood (ν = 10.0±0.2 cm/s using 30° fiber pad). Fiberbot, a low-cost vibration robot equipped with a custom-designed fiber pad with steering and climbing capabilities could be used for studies on collective behavior on a wide range of topographies as well as search and exploratory missions.

pi

DOI [BibTex]

DOI [BibTex]


Platform design and tethered flight of a motor-driven flapping-wing system
Platform design and tethered flight of a motor-driven flapping-wing system

Hines, L., Colmenares, D., Sitti, M.

In Robotics and Automation (ICRA), 2015 IEEE International Conference on, pages: 5838-5845, May 2015 (inproceedings)

Abstract
In this work, we examine two design modifications to a tethered motor-driven flapping-wing system. Previously, we had demonstrated a simple mechanism utilizing a linear transmission for resonant operation and direct drive of the wing flapping angle for control. The initial two-wing system had a weight of 2.7 grams and a maximum lift-to-weight ratio of 1.4. While capable of vertical takeoff, in open-loop flight it demonstrated instability and pitch oscillations at the wing flapping frequency, leading to flight times of only a few wing strokes. Here the effect of vertical wing offset as well as an alternative multi-wing layout is investigated and experimentally tested with newly constructed prototypes. With only a change in vertical wing offset, stable open-loop flight of the two-wing flapping system is shown to be theoretically possible, but difficult to achieve with our current design and operating parameters. Both of the new two and four-wing systems, however, prove capable of flying to the end of the tether, with the four-wing system prototype eliminating disruptive wing beat oscillations.

pi

DOI [BibTex]

DOI [BibTex]


no image
Using IMU Data to Teach a Robot Hand-Clapping Games

Fitter, N. T., Kuchenbecker, K. J.

In Proc. IEEE Haptics Symposium, pages: 353-355, April 2015, Work-in-progress paper. Poster presentation given by Fitter (inproceedings)

hi

[BibTex]

[BibTex]


no image
Haptic Feedback in Transoral Robotic Surgery: A Feasibility Study

Bur, A. M., Gomez, E. D., Rassekh, C. H., Newman, J. G., Weinstein, G. S., Kuchenbecker, K. J.

In Proc. Annual Meeting of the Triological Society at COSM, April 2015, Poster presentation given by Bur (inproceedings)

hi

[BibTex]

[BibTex]


no image
Human Machine Interface for Dexto Eka: - The humanoid robot

Kumra, S., Mohan, M., Gupta, S., Vaswani, H.

In Proceedings of the IEEE International Conference on Robotics, Automation, Control and Embedded Systems (RACE), Chennai, India, Febuary 2015 (inproceedings)

Abstract
This paper illustrates hybrid control system of the humanoid robot, Dexto:Eka: focusing on the dependent or slave mode. Efficiency of any system depends on the fluid operation of its control system. Here, we elucidate the control of 12 DoF robotic arms and an omnidirectional mecanum wheel drive using an exo-frame, and a Graphical User Interface (GUI) and a control column. This paper comprises of algorithms, control mechanisms and overall flow of execution for the regulation of robotic arms, graphical user interface and locomotion.

hi

DOI [BibTex]

DOI [BibTex]


no image
Conception and development of Dexto:Eka: The Humanoid Robot - Part IV

Kumra, S., Mohan, M., Vaswani, H., Gupta, S.

In Proceedings of the IEEE International Conference on Robotics, Automation, Control and Embedded Systems (RACE), Febuary 2015 (inproceedings)

Abstract
This paper elucidates the fourth phase of the development of `Dexto:Eka: - The Humanoid Robot'. It lays special emphasis on the conception of the locomotion drive and the development of vision based system that aids navigation and tele-operation. The first three phases terminated with the completion of two robotic arms with six degrees of freedom each, structural development and the creation of a human machine interface that included an exo-frame, a control column and a graphical user interface. This phase also involved the enhancement of the exo-frame to a vision based system using a Kinect camera. The paper also focuses on the reasons behind choosing the locomotion drive and the benefits it has.

hi

DOI [BibTex]

DOI [BibTex]


no image
Design and Validation of a Practical Simulator for Transoral Robotic Surgery

Bur, A. M., Gomez, E. D., Chalian, A. A., Newman, J. G., Weinstein, G. S., Kuchenbecker, K. J.

In Proc. Society for Robotic Surgery Annual Meeting: Transoral Program, (T8), February 2015, Oral presentation given by Bur (inproceedings)

hi

[BibTex]

[BibTex]


Joint 3D Object and Layout Inference from a single RGB-D Image
Joint 3D Object and Layout Inference from a single RGB-D Image

(Best Paper Award)

Geiger, A., Wang, C.

In German Conference on Pattern Recognition (GCPR), 9358, pages: 183-195, Lecture Notes in Computer Science, Springer International Publishing, 2015 (inproceedings)

Abstract
Inferring 3D objects and the layout of indoor scenes from a single RGB-D image captured with a Kinect camera is a challenging task. Towards this goal, we propose a high-order graphical model and jointly reason about the layout, objects and superpixels in the image. In contrast to existing holistic approaches, our model leverages detailed 3D geometry using inverse graphics and explicitly enforces occlusion and visibility constraints for respecting scene properties and projective geometry. We cast the task as MAP inference in a factor graph and solve it efficiently using message passing. We evaluate our method with respect to several baselines on the challenging NYUv2 indoor dataset using 21 object categories. Our experiments demonstrate that the proposed method is able to infer scenes with a large degree of clutter and occlusions.

avg ps

pdf suppmat video project DOI [BibTex]

pdf suppmat video project DOI [BibTex]


no image
When to use which heuristic: A rational solution to the strategy selection problem

Lieder, F., Griffiths, T. L.

In Proceedings of the 37th Annual Conference of the Cognitive Science Society, 2015 (inproceedings)

Abstract
The human mind appears to be equipped with a toolbox full of cognitive strategies, but how do people decide when to use which strategy? We leverage rational metareasoning to derive a rational solution to this problem and apply it to decision making under uncertainty. The resulting theory reconciles the two poles of the debate about human rationality by proposing that people gradually learn to make rational use of fallible heuristics. We evaluate this theory against empirical data and existing accounts of strategy selection (i.e. SSL and RELACS). Our results suggest that while SSL and RELACS can explain people's ability to adapt to homogeneous environments in which all decision problems are of the same type, rational metareasoning can additionally explain people's ability to adapt to heterogeneous environments and flexibly switch strategies from one decision to the next.

re

link (url) Project Page [BibTex]

link (url) Project Page [BibTex]


Discrete Optimization for Optical Flow
Discrete Optimization for Optical Flow

Menze, M., Heipke, C., Geiger, A.

In German Conference on Pattern Recognition (GCPR), 9358, pages: 16-28, Springer International Publishing, 2015 (inproceedings)

Abstract
We propose to look at large-displacement optical flow from a discrete point of view. Motivated by the observation that sub-pixel accuracy is easily obtained given pixel-accurate optical flow, we conjecture that computing the integral part is the hardest piece of the problem. Consequently, we formulate optical flow estimation as a discrete inference problem in a conditional random field, followed by sub-pixel refinement. Naive discretization of the 2D flow space, however, is intractable due to the resulting size of the label set. In this paper, we therefore investigate three different strategies, each able to reduce computation and memory demands by several orders of magnitude. Their combination allows us to estimate large-displacement optical flow both accurately and efficiently and demonstrates the potential of discrete optimization for optical flow. We obtain state-of-the-art performance on MPI Sintel and KITTI.

avg ps

pdf suppmat project DOI [BibTex]

pdf suppmat project DOI [BibTex]


no image
Children and Adults Differ in their Strategies for Social Learning

Lieder, F., Sim, Z. L., Hu, J. C., Griffiths, T. L., Xu, F.

In Proceedings of the 37th Annual Conference of the Cognitive Science Society, 2015 (inproceedings)

Abstract
Adults and children rely heavily on other people’s testimony. However, domains of knowledge where there is no consensus on the truth are likely to result in conflicting testimonies. Previous research has demonstrated that in these cases, learners look towards the majority opinion to make decisions. However, it remains unclear how learners evaluate social information, given that considering either the overall valence, or the number of testimonies, or both may lead to different conclusions. We therefore formalized several social learning strategies and compared them to the performance of adults and children. We find that children use different strategies than adults. This suggests that the development of social learning may involve the acquisition of cognitive strategies.

re

link (url) [BibTex]

link (url) [BibTex]


Joint 3D Estimation of Vehicles and Scene Flow
Joint 3D Estimation of Vehicles and Scene Flow

Menze, M., Heipke, C., Geiger, A.

In Proc. of the ISPRS Workshop on Image Sequence Analysis (ISA), 2015 (inproceedings)

Abstract
Three-dimensional reconstruction of dynamic scenes is an important prerequisite for applications like mobile robotics or autonomous driving. While much progress has been made in recent years, imaging conditions in natural outdoor environments are still very challenging for current reconstruction and recognition methods. In this paper, we propose a novel unified approach which reasons jointly about 3D scene flow as well as the pose, shape and motion of vehicles in the scene. Towards this goal, we incorporate a deformable CAD model into a slanted-plane conditional random field for scene flow estimation and enforce shape consistency between the rendered 3D models and the parameters of all superpixels in the image. The association of superpixels to objects is established by an index variable which implicitly enables model selection. We evaluate our approach on the challenging KITTI scene flow dataset in terms of object and scene flow estimation. Our results provide a prove of concept and demonstrate the usefulness of our method.

avg ps

PDF [BibTex]

PDF [BibTex]


no image
Learning from others: Adult and child strategies in assessing conflicting ratings

Hu, J., Lieder, F., Griffiths, T. L., Xu, F.

In Biennial Meeting of the Society for Research in Child Development, Philadelphia, Pennsylvania, USA, 2015 (inproceedings)

re

[BibTex]

[BibTex]