Header logo is


2018


no image
Instrumentation, Data, and Algorithms for Visually Understanding Haptic Surface Properties

Burka, A. L.

University of Pennsylvania, Philadelphia, USA, August 2018, Department of Electrical and Systems Engineering (phdthesis)

Abstract
Autonomous robots need to efficiently walk over varied surfaces and grasp diverse objects. We hypothesize that the association between how such surfaces look and how they physically feel during contact can be learned from a database of matched haptic and visual data recorded from various end-effectors' interactions with hundreds of real-world surfaces. Testing this hypothesis required the creation of a new multimodal sensing apparatus, the collection of a large multimodal dataset, and development of a machine-learning pipeline. This thesis begins by describing the design and construction of the Portable Robotic Optical/Tactile ObservatioN PACKage (PROTONPACK, or Proton for short), an untethered handheld sensing device that emulates the capabilities of the human senses of vision and touch. Its sensory modalities include RGBD vision, egomotion, contact force, and contact vibration. Three interchangeable end-effectors (a steel tooling ball, an OptoForce three-axis force sensor, and a SynTouch BioTac artificial fingertip) allow for different material properties at the contact point and provide additional tactile data. We then detail the calibration process for the motion and force sensing systems, as well as several proof-of-concept surface discrimination experiments that demonstrate the reliability of the device and the utility of the data it collects. This thesis then presents a large-scale dataset of multimodal surface interaction recordings, including 357 unique surfaces such as furniture, fabrics, outdoor fixtures, and items from several private and public material sample collections. Each surface was touched with one, two, or three end-effectors, comprising approximately one minute per end-effector of tapping and dragging at various forces and speeds. We hope that the larger community of robotics researchers will find broad applications for the published dataset. Lastly, we demonstrate an algorithm that learns to estimate haptic surface properties given visual input. Surfaces were rated on hardness, roughness, stickiness, and temperature by the human experimenter and by a pool of purely visual observers. Then we trained an algorithm to perform the same task as well as infer quantitative properties calculated from the haptic data. Overall, the task of predicting haptic properties from vision alone proved difficult for both humans and computers, but a hybrid algorithm using a deep neural network and a support vector machine achieved a correlation between expected and actual regression output between approximately ρ = 0.3 and ρ = 0.5 on previously unseen surfaces.

hi

Project Page [BibTex]

2018


Project Page [BibTex]


Thumb xl ar
Robust Visual Augmented Reality in Robot-Assisted Surgery

Forte, M. P.

Politecnico di Milano, Milan, Italy, July 2018, Department of Electronic, Information, and Biomedical Engineering (mastersthesis)

Abstract
The broader research objective of this line of research is to test the hypothesis that real-time stereo video analysis and augmented reality can increase safety and task efficiency in robot-assisted surgery. This master’s thesis aims to solve the first step needed to achieve this goal: the creation of a robust system that delivers the envisioned feedback to a surgeon while he or she controls a surgical robot that is identical to those used on human patients. Several approaches for applying augmented reality to da Vinci Surgical Systems have been proposed, but none of them entirely rely on a clinical robot; specifically, they require additional sensors, depend on access to the da Vinci API, are designed for a very specific task, or were tested on systems that are starkly different from those in clinical use. There has also been prior work that presents the real-world camera view and the computer graphics on separate screens, or not in real time. In other scenarios, the digital information is overlaid manually by the surgeons themselves or by computer scientists, rather than being generated automatically in response to the surgeon’s actions. We attempted to overcome the aforementioned constraints by acquiring input signals from the da Vinci stereo endoscope and providing augmented reality to the console in real time (less than 150 ms delay, including the 62 ms of inherent latency of the da Vinci). The potential benefits of the resulting system are broad because it was built to be general, rather than customized for any specific task. The entire platform is compatible with any generation of the da Vinci System and does not require a dVRK (da Vinci Research Kit) or access to the API. Thus, it can be applied to existing da Vinci Systems in operating rooms around the world.

hi

Project Page [BibTex]

Project Page [BibTex]


no image
Enhanced Non-Steady Gliding Performance of the MultiMo-Bat through Optimal Airfoil Configuration and Control Strategy

Kim, H., Woodward, M. A., Sitti, M.

In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 1382-1388, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Collectives of Spinning Mobile Microrobots for Navigation and Object Manipulation at the Air-Water Interface

Wang, W., Kishore, V., Koens, L., Lauga, E., Sitti, M.

In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 1-9, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Endo-VMFuseNet: A Deep Visual-Magnetic Sensor Fusion Approach for Endoscopic Capsule Robots

Turan, M., Almalioglu, Y., Gilbert, H. B., Sari, A. E., Soylu, U., Sitti, M.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 1-7, 2018 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Endosensorfusion: Particle filtering-based multi-sensory data fusion with switching state-space model for endoscopic capsule robots

Turan, M., Almalioglu, Y., Gilbert, H., Araujo, H., Cemgil, T., Sitti, M.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 1-8, 2018 (inproceedings)

pi

[BibTex]

[BibTex]

2014


Thumb xl publications toc
Geckogripper: A soft, inflatable robotic gripper using gecko-inspired elastomer micro-fiber adhesives

Song, S., Majidi, C., Sitti, M.

In Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ International Conference on, pages: 4624-4629, September 2014 (inproceedings)

Abstract
This paper proposes GeckoGripper, a novel soft, inflatable gripper based on the controllable adhesion mechanism of gecko-inspired micro-fiber adhesives, to pick-and-place complex and fragile non-planar or planar parts serially or in parallel. Unlike previous fibrillar structures that use peel angle to control the manipulation of parts, we developed an elastomer micro-fiber adhesive that is fabricated on a soft, flexible membrane, increasing the adaptability to non-planar three-dimensional (3D) geometries and controllability in adhesion. The adhesive switching ratio (the ratio between the maximum and minimum adhesive forces) of the developed gripper was measured to be around 204, which is superior to previous works based on peel angle-based release control methods. Adhesion control mechanism based on the stretch of the membrane and superior adaptability to non-planar 3D geometries enable the micro-fibers to pick-and-place various 3D parts as shown in demonstrations.

pi

DOI [BibTex]

2014


DOI [BibTex]


no image
Three-dimensional robotic manipulation and transport of micro-scale objects by a magnetically driven capillary micro-gripper

Giltinan, J., Diller, E., Mayda, C., Sitti, M.

In Robotics and Automation (ICRA), 2014 IEEE International Conference on, pages: 2077-2082, 2014 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Robotic assembly of hydrogels for tissue engineering and regenerative medicine

Tasoglu, S, Diller, E, Guven, S, Sitti, M, Demirci, U

In Journal of Tissue Engineering and Regenerative Medicine, 8, pages: 181-182, 2014 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Versatile non-contact micro-manipulation method using rotational flows locally induced by magnetic microrobots

Ye, Z., Edington, C., Russell, A. J., Sitti, M.

In Advanced Intelligent Mechatronics (AIM), 2014 IEEE/ASME International Conference on, pages: 26-31, 2014 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Structural optimization method towards synthesis of small scale flexure-based mobile grippers

Lum, G. Z., Diller, E., Sitti, M.

In Robotics and Automation (ICRA), 2014 IEEE International Conference on, pages: 2339-2344, 2014 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Six-Degrees-of-Freedom Remote Actuation of Magnetic Microrobots.

Diller, E. D., Giltinan, J., Lum, G. Z., Ye, Z., Sitti, M.

In Robotics: Science and Systems, 2014 (inproceedings)

pi

[BibTex]

[BibTex]

2012


no image
Topological optimization for continuum compliant mechanisms via morphological evolution of traditional mechanisms

Lum, GZ, Yeo, SH, Yang, GL, Teo, TJ, Sitti, M

In 4th International Conference on Computational Methods, pages: 8, 2012 (inproceedings)

pi

[BibTex]

2012


[BibTex]


no image
Flapping Wings with DC-Motors via Direct, Elastic Transmissions

Azhar, M., Campolo, D., Lau, G., Sitti, M.

In Proceedings of International Conference on Intelligent Unmanned Systems, 8, 2012 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Investigation of bioinspired gecko fibers to improve adhesion of HeartLander surgical robot

Tortora, G., Glass, P., Wood, N., Aksak, B., Menciassi, A., Sitti, M., Riviere, C.

In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE, pages: 908-911, 2012 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Magnetic hysteresis for multi-state addressable magnetic microrobotic control

Diller, E., Miyashita, S., Sitti, M.

In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages: 2325-2331, 2012 (inproceedings)

pi

[BibTex]

[BibTex]

1998


no image
Nano tele-manipulation using virtual reality interface

Sitti, M., Horiguchi, S., Hashimoto, H.

In Industrial Electronics, 1998. Proceedings. ISIE’98. IEEE International Symposium on, 1, pages: 171-176, 1998 (inproceedings)

pi

[BibTex]

1998


[BibTex]


no image
Tele-nanorobotics using atomic force microscope

Sitti, M., Hashimoto, H.

In Intelligent Robots and Systems, 1998. Proceedings., 1998 IEEE/RSJ International Conference on, 3, pages: 1739-1746, 1998 (inproceedings)

pi

[BibTex]

[BibTex]


no image
2D micro particle assembly using atomic force microscope

Sitti, M., Hirahara, K., Hashimoto, H.

In Micromechatronics and Human Science, 1998. MHS’98. Proceedings of the 1998 International Symposium on, pages: 143-148, 1998 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Macro to nano tele-manipulation through nanoelectromechanical systems

Sitti, M., Hashimoto, H.

In Industrial Electronics Society, 1998. IECON’98. Proceedings of the 24th Annual Conference of the IEEE, 1, pages: 98-103, 1998 (inproceedings)

pi

[BibTex]

[BibTex]