Intuitive Social-Physical Robots for Exercise

Institute Homepage

Institute Homepage Sign In

Back

Research Overview

Natural Embodied Touch

Finger-Surface Contact Mechanics in Diverse Moisture Conditions

Perceptual Integration of Contact Force Components During Fingertip Sliding

Material Intelligence of Animal Whiskers

Artificial Touch Sensing and Processing

Giving Touch to Soft Robot Fingertips Using Vision, Audio and Machine Learning

Capturing and Recognizing Multimodal Surface Interactions

Multimodal Puncture Detection for Urgent Percutaneous Therapies

Haptic Actuation

Quantifying the Quality of Haptic Interfaces

Shape-Changing Haptic Interfaces

Generating Clear Vibrotactile Cues with Magnets Embedded in a Soft Finger Sheath

Salient Full-Fingertip Haptic Feedback Enabled by Wearable Electrohydraulic Actuation

Cutaneous Electrohydraulic (CUTE) Wearable Devices for Pleasant Broad-Bandwidth Haptic Cues

Modeling Finger-Touchscreen Contact during Electrovibration

Perception of Ultrasonic Friction Pulses

CAPT Motor: A Two-Phase Ironless Motor Structure

Immersive Teleoperation

4D Intraoperative Surgical Perception: Anatomical Shape Reconstruction from Multiple Viewpoints

Visual-Inertial Force Estimation in Robotic Surgery

Enhancing Robotic Surgical Training

AiroTouch: Naturalistic Vibrotactile Feedback for Large-Scale Telerobotic Assembly

Optimization-Based Whole-Arm Teleoperation for Natural Human-Robot Interaction

Human-Robot Interaction

Enhancing HRI through Responsive Multimodal Feedback

Haptic Empathetic Robot Animal (HERA)

How does HuggieBot compare to a human hugging partner?

HuggieBot: Evolution of an Interactive Hugging Robot with Visual and Haptic Perception

Human Movement

Reconstructing Sign-Language Movements from Images and Bioimpedance Measurements

Advancing Gait-Retraining Techniques

Multimodal Sensing Reveals the Dynamics of Time-Constrained Teamwork

Previous HI Projects

Finger-Surface Contact Mechanics in Diverse Moisture Conditions

Computational Modeling of Finger-Surface Contact

Perceptual Integration of Contact Force Components During Tactile Stimulation

Dynamic Models and Wearable Tactile Devices for the Fingertips

Novel Designs and Rendering Algorithms for Fingertip Haptic Devices

Dimensional Reduction from 3D to 1D for Realistic Vibration Rendering

Prendo: Analyzing Human Grasping Strategies for Visually Occluded Objects

Learning Upper-Limb Exercises from Demonstrations

Minimally Invasive Surgical Training with Multimodal Feedback and Automatic Skill Evaluation

Efficient Large-Area Tactile Sensing for Robot Skin

Haptic Feedback and Autonomous Reflexes for Upper-limb Prostheses

Gait Retraining

Modeling Hand Deformations During Contact

Intraoperative AR Assistance for Robot-Assisted Minimally Invasive Surgery

Immersive VR for Phantom Limb Pain

Visual and Haptic Perception of Real Surfaces

Haptipedia

Gait Propulsion Trainer

TouchTable: A Musical Interface with Haptic Feedback for DJs

Exercise Games with Baxter

Intuitive Social-Physical Robots for Exercise

How Should Robots Hug?

Hierarchical Structure for Learning from Demonstration

Fabrication of HuggieBot 2.0: A More Huggable Robot

Learning Haptic Adjectives from Tactile Data

Feeling With Your Eyes: Visual-Haptic Surface Interaction

S-BAN

General Tactile Sensor Model

Insight: a Haptic Sensor Powered by Vision and Machine Learning

Haptic Intelligence Members Publications

Intuitive Social-Physical Robots for Exercise

Sab2022 v3 small — We are creating and evaluating ways in which robots can get people to do physical activity. (a, b) An end-user playing two of our eight exercise games with Baxter. (c) An operator teleoperating Baxter's arms to give a high-five to an end-user. (d) Baxter imitating an operator's head movement and facial expression. (e, f) An end-user interacting with Baxter within the Robot Interaction Studio.

Physical activity plays a critical role in maintaining one's health. Past research has shown that well-designed robots can motivate users to perform regular exercise and physical therapy.

To understand how people respond to exercise-based interactions with a robot, we developed eight exercise games for the Rethink Robotics Baxter Research Robot. These games were developed with the input and guidance of experts in game design, therapy, and rehabilitation, and via extensive pilot testing []. Results from our game evaluation study with 20 younger and 20 older adults support the potential use of bimanual humanoid robots for social-physical exercise interactions []. However, these robots must have customizable behaviors and end-user monitoring capabilities to be viable in real-world scenarios.

Novel robot behaviors can be created using teleoperation. We are creating an interface that allows a therapist operator to control Baxter in a physically and emotionally expressive manner []. Baxter's arm and head movements are based on measurements from Xsens, an inertial motion-capture suit worn by the operator. Ongoing work centers on optimization-based kinematic retargeting in real time. The robot's head and face can simultaneously be teleoperated via emotion recognition on a video of the operator.

The end-users of our robotic exercise coach will be monitored via the Robot Interaction Studio, a platform for enabling minimally supervised human-robot interaction. This system combines Captury Live, a real-time markerless motion-capture system, with a ROS-compatible robot to estimate user actions in real time and provide corrective feedback []. We evaluated this platform via a user study where Baxter sequentially presented the user with three gesture-based cues in randomized order []. Without instructions, we found that the users tended to explore the interaction workspace, mimic Baxter, and interact with Baxter's hands.

Our next step is to enable experts in exercise therapy to prototype a wide range of social-physical interactions for Baxter using our teleoperation interface []. These teleoperated behaviors can be recorded and then autonomously learned and repeated. We envision scenarios in a community rehabilitation center where a robot in the Robot Interaction Studio acts as an exercise partner or coach for an end-user.

Members

Haptic Intelligence

Mayumi Mohan

Research Scientist

Haptic Intelligence

Haliza MatHusin

Research Scientist

Haptic Intelligence

Cara Nunez

Visitor

Haptic Intelligence

Katherine J. Kuchenbecker

Director

Haptic Intelligence

Coco Langens

Intern

Publications

Haptic Intelligence Conference Paper Robot Interaction Studio: A Platform for Unsupervised HRI Mohan, M., Nunez, C. M., Kuchenbecker, K. J. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 3330-3336, Xi’an, China, May 2021 (Published)

Abstract ›

Robots hold great potential for supporting exercise and physical therapy, but such systems are often cumbersome to set up and require expert supervision. We aim to solve these concerns by combining Captury Live, a real-time markerless motion-capture system, with a Rethink Robotics Baxter Research Robot to create the Robot Interaction Studio. We evaluated this platform for unsupervised human-robot interaction (HRI) through a 75-minute-long user study with seven adults who were given minimal instructions and no feedback about their actions. The robot used sounds, facial expressions, facial colors, head motions, and arm motions to sequentially present three categories of cues in randomized order while constantly rotating its face screen to look at the user. Analysis of the captured user motions shows that the cue type significantly affected the distance subjects traveled and the amount of time they spent within the robot’s reachable workspace, in alignment with the design of the cues. Heat map visualizations of the recorded user hand positions confirm that users tended to mimic the robot’s arm poses. Despite some initial frustration, taking part in this study did not significantly change user opinions of the robot. We reflect on the advantages of the proposed approach to unsupervised HRI as well as the limitations and possible future extensions of our system.

DOI BibTeX

Haptic Intelligence Miscellaneous Evaluation of a Teleoperated Robotic Exercise Coach Mohan, M., Mat Husin, H., Kuchenbecker, K. J. Workshop paper (4 pages) presented at the HRI Workshop on Workshop YOUR study design! Participatory critique and refinement of participants’ studies, Virtual, March 2021 (Published) BibTeX

Haptic Intelligence Miscellaneous How Does Real-Time Feedback Affect Communicative Actions in Social-Physical Human-Robot Interaction? Mohan, M., Nunez, C. M., Kuchenbecker, K. J. Workshop paper (2 pages) presented at the ROMAN Workshop on Quality of Interaction in Socially Assistive Robots (QISAR), Virtual, August 2020 (Published)

Abstract ›

Social robots are becoming more common, especially to motivate older adults to exercise and stay healthy. To increase the effectiveness of such robots, researchers need to develop autonomous interactions that are understandable to the user without help from a human operator. Motivated by this requirement, we have programmed a Baxter robot to play an exercise game via multi-modal non-verbal communication. When the user is confused or makes a mistake, the robot can optionally provide corrective feedback based on real-time measurements of user actions. We hypothesize feedback will improve both the user's physical performance and the user's opinion of the robot's social skills during a planned experiment.

BibTeX

Haptic Intelligence Article Exercising with Baxter: Preliminary Support for Assistive Social-Physical Human-Robot Interaction Fitter, N. T., Mohan, M., Kuchenbecker, K. J., Johnson, M. J. Journal of NeuroEngineering and Rehabilitation, 17:1-22, February 2020 (Published)

Abstract ›

Background: The worldwide population of older adults will soon exceed the capacity of assisted living facilities. Accordingly, we aim to understand whether appropriately designed robots could help older adults stay active at home. Methods: Building on related literature as well as guidance from experts in game design, rehabilitation, and physical and occupational therapy, we developed eight human-robot exercise games for the Baxter Research Robot, six of which involve physical human-robot contact. After extensive iteration, these games were tested in an exploratory user study including 20 younger adult and 20 older adult users. Results: Only socially and physically interactive games fell in the highest ranges for pleasantness, enjoyment, engagement, cognitive challenge, and energy level. Our games successfully spanned three different physical, cognitive, and temporal challenge levels. User trust and confidence in Baxter increased significantly between pre- and post-study assessments. Older adults experienced higher exercise, energy, and engagement levels than younger adults, and women rated the robot more highly than men on several survey questions. Conclusions: The results indicate that social-physical exercise with a robot is more pleasant, enjoyable, engaging, cognitively challenging, and energetic than similar interactions that lack physical touch. In addition to this main finding, researchers working in similar areas can build on our design practices, our open-source resources, and the age-group and gender differences that we found.

DOI BibTeX

Haptic Intelligence Miscellaneous A Design Tool for Therapeutic Social-Physical Human-Robot Interactions Mohan, M., Kuchenbecker, K. J. 727-729, Workshop paper (3 pages) presented at the HRI Pioneers Workshop, Daegu, South Korea, March 2019 (Published)

Abstract ›

We live in an aging society; social-physical human-robot interaction has the potential to keep our elderly adults healthy by motivating them to exercise. After summarizing prior work, this paper proposes a tool that can be used to design exercise and therapy interactions to be performed by an upper-body humanoid robot. The interaction design tool comprises a teleoperation system that transmits the operator’s arm motions, head motions and facial expression along with an interface to monitor and assess the motion of the user interacting with the robot. We plan to use this platform to create dynamic and intuitive exercise interactions.

DOI BibTeX

Haptic Intelligence Miscellaneous Exercising with Baxter: Design and Evaluation of Assistive Social-Physical Human-Robot Interaction Fitter, N. T., Mohan, M., Kuchenbecker, K. J., Johnson, M. J. Workshop paper (6 pages) presented at the HRI Workshop on Personal Robots for Exercising and Coaching, Chicago, USA, March 2018 (Published)

Abstract ›

The worldwide population of older adults is steadily increasing and will soon exceed the capacity of assisted living facilities. Accordingly, we aim to understand whether appropriately designed robots could help older adults stay active and engaged while living at home. We developed eight human-robot exercise games for the Baxter Research Robot with the guidance of experts in game design, therapy, and rehabilitation. After extensive iteration, these games were employed in a user study that tested their viability with 20 younger and 20 older adult users. All participants were willing to enter Baxter’s workspace and physically interact with the robot. User trust and confidence in Baxter increased significantly between pre- and post-experiment assessments, and one individual from the target user population supplied us with abundant positive feedback about her experience. The preliminary results presented in this paper indicate potential for the use of two-armed human-scale robots for social-physical exercise interaction.

URL BibTeX

Haptic Intelligence Miscellaneous Physically Interactive Exercise Games with a Baxter Robot Fitter, N. T., Kuchenbecker, K. J. Hands-on demonstration presented at the IEEE World Haptics Conference (WHC), Munich, Germany, June 2017 (Published) BibTeX

Haptic Intelligence Miscellaneous Hand-Clapping Games with a Baxter Robot Fitter, N. T., Kuchenbecker, K. J. Hands-on demonstration presented at the ACM/IEEE International Conference on Human-Robot Interaction (HRI), Vienna, Austria, March 2017 (Published)

Abstract ›

Robots that work alongside humans might be more effective if they could forge a strong social bond with their human partners. Hand-clapping games and other forms of rhythmic social-physical interaction may foster human-robot teamwork, but the design of such interactions has scarcely been explored. At the HRI 2017 conference, we will showcase several such interactions taken from our recent work with the Rethink Robotics Baxter Research Robot, including tempo-matching, Simon says, and Pat-a-cake-like games. We believe conference attendees will be both entertained and intrigued by this novel demonstration of social-physical HRI.

BibTeX

Haptic Intelligence Miscellaneous Designing Human-Robot Exercise Games for Baxter Fitter, N. T., Hawkes, D. T., Johnson, M. J., Kuchenbecker, K. J. Daejeon, South Korea, October 2016, Late-breaking results report presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Published) BibTeX