Publications | Max Planck Institute for Intelligent Systems

71 results (View BibTeX file of all listed publications)

2018

Instrumentation, Data, and Algorithms for Visually Understanding Haptic Surface Properties

University of Pennsylvania, Philadelphia, USA, August 2018, Department of Electrical and Systems Engineering (phdthesis)

Abstract

Autonomous robots need to efficiently walk over varied surfaces and grasp diverse objects. We hypothesize that the association between how such surfaces look and how they physically feel during contact can be learned from a database of matched haptic and visual data recorded from various end-effectors' interactions with hundreds of real-world surfaces. Testing this hypothesis required the creation of a new multimodal sensing apparatus, the collection of a large multimodal dataset, and development of a machine-learning pipeline. This thesis begins by describing the design and construction of the Portable Robotic Optical/Tactile ObservatioN PACKage (PROTONPACK, or Proton for short), an untethered handheld sensing device that emulates the capabilities of the human senses of vision and touch. Its sensory modalities include RGBD vision, egomotion, contact force, and contact vibration. Three interchangeable end-effectors (a steel tooling ball, an OptoForce three-axis force sensor, and a SynTouch BioTac artificial fingertip) allow for different material properties at the contact point and provide additional tactile data. We then detail the calibration process for the motion and force sensing systems, as well as several proof-of-concept surface discrimination experiments that demonstrate the reliability of the device and the utility of the data it collects. This thesis then presents a large-scale dataset of multimodal surface interaction recordings, including 357 unique surfaces such as furniture, fabrics, outdoor fixtures, and items from several private and public material sample collections. Each surface was touched with one, two, or three end-effectors, comprising approximately one minute per end-effector of tapping and dragging at various forces and speeds. We hope that the larger community of robotics researchers will find broad applications for the published dataset. Lastly, we demonstrate an algorithm that learns to estimate haptic surface properties given visual input. Surfaces were rated on hardness, roughness, stickiness, and temperature by the human experimenter and by a pool of purely visual observers. Then we trained an algorithm to perform the same task as well as infer quantitative properties calculated from the haptic data. Overall, the task of predicting haptic properties from vision alone proved difficult for both humans and computers, but a hybrid algorithm using a deep neural network and a support vector machine achieved a correlation between expected and actual regression output between approximately ρ = 0.3 and ρ = 0.5 on previously unseen surfaces.

Project Page [BibTex]

2018

hi Burka, A. L. Instrumentation, Data, and Algorithms for Visually Understanding Haptic Surface Properties University of Pennsylvania, Philadelphia, USA, August 2018, Department of Electrical and Systems Engineering (phdthesis)

Project Page [BibTex]

Robust Visual Augmented Reality in Robot-Assisted Surgery

Forte, M. P.

Politecnico di Milano, Milan, Italy, July 2018, Department of Electronic, Information, and Biomedical Engineering (mastersthesis)

Abstract

The broader research objective of this line of research is to test the hypothesis that real-time stereo video analysis and augmented reality can increase safety and task efficiency in robot-assisted surgery. This master’s thesis aims to solve the first step needed to achieve this goal: the creation of a robust system that delivers the envisioned feedback to a surgeon while he or she controls a surgical robot that is identical to those used on human patients. Several approaches for applying augmented reality to da Vinci Surgical Systems have been proposed, but none of them entirely rely on a clinical robot; specifically, they require additional sensors, depend on access to the da Vinci API, are designed for a very specific task, or were tested on systems that are starkly different from those in clinical use. There has also been prior work that presents the real-world camera view and the computer graphics on separate screens, or not in real time. In other scenarios, the digital information is overlaid manually by the surgeons themselves or by computer scientists, rather than being generated automatically in response to the surgeon’s actions. We attempted to overcome the aforementioned constraints by acquiring input signals from the da Vinci stereo endoscope and providing augmented reality to the console in real time (less than 150 ms delay, including the 62 ms of inherent latency of the da Vinci). The potential benefits of the resulting system are broad because it was built to be general, rather than customized for any specific task. The entire platform is compatible with any generation of the da Vinci System and does not require a dVRK (da Vinci Research Kit) or access to the API. Thus, it can be applied to existing da Vinci Systems in operating rooms around the world.

Project Page [BibTex]

hi Forte, M. P. Robust Visual Augmented Reality in Robot-Assisted Surgery Politecnico di Milano, Milan, Italy, July 2018, Department of Electronic, Information, and Biomedical Engineering (mastersthesis)

Project Page [BibTex]

Colloidal Chemical Nanomotors

Alarcon-Correa, M.

Colloidal Chemical Nanomotors, pages: 150, Cuvillier Verlag, MPI-IS , June 2018 (phdthesis)

Abstract

Synthetic sophisticated nanostructures represent a fundamental building block for the development of nanotechnology. The fabrication of nanoparticles complex in structure and material composition is key to build nanomachines that can operate as man-made nanoscale motors, which autonomously convert external energy into motion. To achieve this, asymmetric nanoparticles were fabricated combining a physical vapor deposition technique known as NanoGLAD and wet chemical synthesis. This thesis primarily concerns three complex colloidal systems that have been developed: i)Hollow nanocup inclusion complexes that have a single Au nanoparticle in their pocket. The Au particle can be released with an external trigger. ii)The smallest self-propelling nanocolloids that have been made to date, which give rise to a local concentration gradient that causes enhanced diffusion of the particles. iii)Enzyme-powered pumps that have been assembled using bacteriophages as biological nanoscaffolds. This construct also can be used for enzyme recovery after heterogeneous catalysis.

MPI Papers

Departments

Research Groups

Publication Type

Year

2018

2018

2016

2016

2010

2010

Latest News

Links

Contact Us