Institute Homepage
Institute Homepage DE Sign In

Back

Autonomous Motion Conference Paper 2018

Motion-based Object Segmentation based on Dense RGB-D Scene Flow

Project Page arXiv
Thumb ticker sm me portrait
Autonomous Motion
Jeannette Bohg
  • Affiliated Researcher
Screen shot 2018 04 19 at 14.57.08

Given two consecutive RGB-D images, we propose a model that estimates a dense 3D motion field, also known as scene flow. We take advantage of the fact that in robot manipulation scenarios, scenes often consist of a set of rigidly moving objects. Our model jointly estimates (i) the segmentation of the scene into an unknown but finite number of objects, (ii) the motion trajectories of these objects and (iii) the object scene flow. We employ an hourglass, deep neural network architecture. In the encoding stage, the RGB and depth images undergo spatial compression and correlation. In the decoding stage, the model outputs three images containing a per-pixel estimate of the corresponding object center as well as object translation and rotation. This forms the basis for inferring the object segmentation and final object scene flow. To evaluate our model, we generated a new and challenging, large-scale, synthetic dataset that is specifically targeted at robotic manipulation: It contains a large number of scenes with a very diverse set of simultaneously moving 3D objects and is recorded with a commonly-used RGB-D camera. In quantitative experiments, we show that we significantly outperform state-of-the-art scene flow and motion-segmentation methods. In qualitative experiments, we show how our learned model transfers to challenging real-world scenes, visually generating significantly better results than existing methods.

Author(s): Lin Shao and Parth Shah and Vikranth Dwaracherla and Jeannette Bohg
Links:
Book Title: IEEE Robotics and Automation Letters
Volume: 3
Number (issue): 4
Pages: 3797-3804
Year: 2018
Month: October
Day: 14
Publisher: IEEE
Bibtex Type: Conference Paper (conference)
DOI: 10.1109/LRA.2018.2856525
Event Name: IEEE/RSJ International Conference on Intelligent Robots and Systems
Event Place: Madrid, Spain
Electronic Archiving: grant_archive

BibTex 

@conference{2018_RAL_ssdb,
  title = {Motion-based Object Segmentation based on Dense RGB-D Scene Flow},
  booktitle = {IEEE Robotics and Automation Letters},
  abstract = {Given two consecutive RGB-D images, we propose a model that estimates a dense 3D motion field, also known as scene flow. We take advantage of the fact that in robot manipulation scenarios, scenes often consist of a set of rigidly moving objects. Our model jointly estimates (i) the segmentation of the scene into an unknown but finite number of objects, (ii) the motion trajectories of these objects and (iii) the object scene flow. We employ an hourglass, deep neural network architecture. In the encoding stage, the RGB and depth images undergo spatial compression and correlation. In the decoding stage, the model outputs three images containing a per-pixel estimate of the corresponding object center as well as object translation and rotation. This forms the basis for inferring the object segmentation and final object scene flow. To evaluate our model, we generated a new and challenging, large-scale, synthetic dataset that is specifically targeted at robotic manipulation: It contains a large number of scenes with a very diverse set of simultaneously moving 3D objects and is recorded with a commonly-used RGB-D camera. In quantitative experiments, we show that we significantly outperform state-of-the-art scene flow and motion-segmentation methods. In qualitative experiments, we show how our learned model transfers to challenging real-world scenes, visually generating significantly better results than existing methods. },
  volume = {3},
  number = {4},
  pages = {3797-3804},
  publisher = {IEEE},
  month = oct,
  year = {2018},
  slug = {2018_ral_ssdb},
  author = {Shao, Lin and Shah, Parth and Dwaracherla, Vikranth and Bohg, Jeannette},
  month_numeric = {10}
}