Institute Homepage
Institute Homepage Sign In

Back

Research Overview

Natural Embodied Touch

Finger-Surface Contact Mechanics in Diverse Moisture Conditions

Perceptual Integration of Contact Force Components During Fingertip Sliding

Material Intelligence of Animal Whiskers

Artificial Touch Sensing and Processing

Giving Touch to Soft Robot Fingertips Using Vision, Audio and Machine Learning

Capturing and Recognizing Multimodal Surface Interactions

Multimodal Puncture Detection for Urgent Percutaneous Therapies

Haptic Actuation

Quantifying the Quality of Haptic Interfaces

Shape-Changing Haptic Interfaces

Generating Clear Vibrotactile Cues with Magnets Embedded in a Soft Finger Sheath

Salient Full-Fingertip Haptic Feedback Enabled by Wearable Electrohydraulic Actuation

Cutaneous Electrohydraulic (CUTE) Wearable Devices for Pleasant Broad-Bandwidth Haptic Cues

Modeling Finger-Touchscreen Contact during Electrovibration

Perception of Ultrasonic Friction Pulses

CAPT Motor: A Two-Phase Ironless Motor Structure

Immersive Teleoperation

4D Intraoperative Surgical Perception: Anatomical Shape Reconstruction from Multiple Viewpoints

Visual-Inertial Force Estimation in Robotic Surgery

Enhancing Robotic Surgical Training

AiroTouch: Naturalistic Vibrotactile Feedback for Large-Scale Telerobotic Assembly

Optimization-Based Whole-Arm Teleoperation for Natural Human-Robot Interaction

Human-Robot Interaction

Enhancing HRI through Responsive Multimodal Feedback

Haptic Empathetic Robot Animal (HERA)

How does HuggieBot compare to a human hugging partner?

HuggieBot: Evolution of an Interactive Hugging Robot with Visual and Haptic Perception

Human Movement

Reconstructing Sign-Language Movements from Images and Bioimpedance Measurements

Advancing Gait-Retraining Techniques

Multimodal Sensing Reveals the Dynamics of Time-Constrained Teamwork

Previous HI Projects

Finger-Surface Contact Mechanics in Diverse Moisture Conditions

Perceptual Integration of Contact Force Components During Tactile Stimulation

Dynamic Models and Wearable Tactile Devices for the Fingertips

Novel Designs and Rendering Algorithms for Fingertip Haptic Devices

Dimensional Reduction from 3D to 1D for Realistic Vibration Rendering

Prendo: Analyzing Human Grasping Strategies for Visually Occluded Objects

Learning Upper-Limb Exercises from Demonstrations

Minimally Invasive Surgical Training with Multimodal Feedback and Automatic Skill Evaluation

Efficient Large-Area Tactile Sensing for Robot Skin

Haptic Feedback and Autonomous Reflexes for Upper-limb Prostheses

Gait Retraining

Intraoperative AR Assistance for Robot-Assisted Minimally Invasive Surgery

Haptipedia

Exercise Games with Baxter

Intuitive Social-Physical Robots for Exercise

Insight: a Haptic Sensor Powered by Vision and Machine Learning

Haptic Intelligence Members Publications

Haptic Feedback and Autonomous Reflexes for Upper-limb Prostheses

The studied task required the user to pick up and move the metal cylinder using a myoelectric prosthesis in the absence of direct vision. (a) Ottobock SensorHand Speed featuring custom-built contact-location and pressure sensors. (b) Experimental setup including an adapter and counterbalance weights to allow use by non-amputee individuals. (c) Traces of select signals from a participant who received vibrotactile feedback and used the prosthesis with reflexes enabled. From top to bottom: raw and reflex-modulated motor command to close the prosthetic hand, hand aperture, pressure sensor, contact-location sensor, current to vibrotactor, and lastly, the distance of the object to the end bin (purple line) and height of the object (pink line).

Upper-limb loss prevents amputees from being able to carry out many day-to-day tasks that are necessary for ensuring a good quality of life. Although commercially available prosthetic limbs can replace some of the lost motor function of the healthy limb, the lack of sensory feedback causes amputees to rely heavily on vision to estimate haptic information like contact, slip, and stiffness. This visual crutch prevents object manipulation in a variety of scenarios that healthy individuals take for granted. For example, being able to pick up a pen from a desk while watching a lecture is a task amputees would be unable to do due to a lack of haptic sensation.

For a prosthesis user to pick up an object without direct vision, we hypothesized that they would need to know where on their prosthetic hand they are touching the object, so that they can appropriately adjust the orientation and placement of their hand. We further hypothesized that imbuing a prosthesis with its own reflexes to prevent adverse events like object slip or excessive grasping force can also improve the performance of dexterous tasks. To realize such a system, we built a pressure sensor for use in the autonomous control loop and developed a novel contact-location sensor. This sensor’s signals provide a continuous indication of single-site contact location, which can be fed back to the user in a variety of ways, such as through vibrotactile feedback or distributed pressure feedback.

We investigated the ability of participants to use a myoelectric prosthesis in a reach-to-pick-and-place task without using direct vision.  Several combinations of haptic feedback and reflex controllers were tested. Initial results comparing the standard myoelectric prosthesis to a prosthesis with reflex control and vibrotactile feedback show that the combination of tactile feedback and control significantly improve performance consistency [File Icon].

Members

Haptic Intelligence
Neha Thomas
  • Visitor
Haptic Intelligence
Farimah Fazlollahi
  • Postdoctoral Researcher
Haptic Intelligence
Katherine J. Kuchenbecker
Director

Publications

Haptic Intelligence Article The Utility of Synthetic Reflexes and Haptic Feedback for Upper-Limb Prostheses in a Dexterous Task Without Direct Vision Thomas, N., Fazlollahi, F., Kuchenbecker, K. J., Brown, J. D. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 31:169-179, January 2023 (Published)
Individuals who use myoelectric upper-limb prostheses often rely heavily on vision to complete their daily activities. They thus struggle in situations where vision is overloaded, such as multitasking, or unavailable, such as poor lighting conditions. Able-bodied individuals can easily accomplish such tasks due to tactile reflexes and haptic sensation guiding their upper-limb motor coordination. Based on these principles, we developed and tested two novel prosthesis systems that incorporate autonomous controllers and provide the user with touch-location feedback through either vibration or distributed pressure. These capabilities were made possible by installing a custom contact-location sensor on the fingers of a commercial prosthetic hand, along with a custom pressure sensor on the thumb. We compared the performance of the two systems against a standard myoelectric prosthesis and a myoelectric prosthesis with only autonomous controllers in a difficult reach-to-pick-and-place task conducted without direct vision. Results from 40 able-bodied participants in this between-subjects study indicated that vibrotactile feedback combined with synthetic reflexes proved significantly more advantageous than the standard prosthesis in several of the task milestones. In addition, vibrotactile feedback and synthetic reflexes improved grasp placement compared to only synthetic reflexes or pressure feedback combined with synthetic reflexes. These results indicate that autonomous controllers and haptic feedback together facilitate success in dexterous tasks without vision, and that the type of haptic display matters.
DOI BibTeX
Haptic Intelligence Conference Paper Sensorimotor-Inspired Tactile Feedback and Control Improve Consistency of Prosthesis Manipulation in the Absence of Direct Vision Thomas, N., Fazlollahi, F., Brown, J. D., Kuchenbecker, K. J. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 6174-6181, Prague, Czech Republic, September 2021 (Published)
The lack of haptically aware upper-limb prostheses forces amputees to rely largely on visual cues to complete activities of daily living. In contrast, non-amputees inherently rely on conscious haptic perception and automatic tactile reflexes to govern volitional actions in situations that do not allow for constant visual attention. We therefore propose a myoelectric prosthesis system that reflects these concepts to aid manipulation performance without direct vision. To implement this design, we constructed two fabric-based tactile sensors that measure contact location along the palmar and dorsal sides of the prosthetic fingers and grasp pressure at the tip of the prosthetic thumb. Inspired by the natural sensorimotor system, we use the measurements from these sensors to provide vibrotactile feedback of contact location and implement a tactile grasp controller with reflexes that prevent over-grasping and object slip. We compare this tactile system to a standard myoelectric prosthesis in a challenging reach-to-pick-and-place task conducted without direct vision; 17 non-amputee adults took part in this single-session between-subjects study. Participants in the tactile group achieved more consistent high performance compared to participants in the standard group. These results show that adding contact-location feedback and reflex control increases the consistency with which objects can be grasped and moved without direct vision in upper-limb prosthetics
DOI BibTeX