Header logo is

Fingertip Interaction Metrics Correlate with Visual and Haptic Perception of Real Surfaces


Conference Paper


Both vision and touch contribute to the perception of real surfaces. Although there have been many studies on the individual contributions of each sense, it is still unclear how each modality’s information is processed and integrated. To fill this gap, we investigated the similarity of visual and haptic perceptual spaces, as well as how well they each correlate with fingertip interaction metrics. Twenty participants interacted with ten different surfaces from the Penn Haptic Texture Toolkit by either looking at or touching them and judged their similarity in pairs. By analyzing the resulting similarity ratings using multi-dimensional scaling (MDS), we found that surfaces are similarly organized within the three-dimensional perceptual spaces of both modalities. Also, between-participant correlations were significantly higher in the haptic condition. In a separate experiment, we obtained the contact forces and accelerations acting on one finger interacting with each surface in a controlled way. We analyzed the collected fingertip interaction data in both the time and frequency domains. Our results suggest that the three perceptual dimensions for each modality can be represented by roughness/smoothness, hardness/softness, and friction, and that these dimensions can be estimated by surface vibration power, tap spectral centroid, and kinetic friction coefficient, respectively.

Author(s): Yasemin Vardar and Christian Wallraven and Katherine J. Kuchenbecker
Book Title: Proceedings of the IEEE World Haptics Conference (WHC)
Year: 2019
Month: July

Department(s): Haptic Intelligence
Research Project(s): Visual and Haptic Perception of Real Surfaces
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Address: Tokyo, Japan
State: Accepted


  title = {Fingertip Interaction Metrics Correlate with Visual and Haptic Perception of Real Surfaces},
  author = {Vardar, Yasemin and Wallraven, Christian and Kuchenbecker, Katherine J.},
  booktitle = {Proceedings of the IEEE World Haptics Conference (WHC)},
  address = {Tokyo, Japan},
  month = jul,
  year = {2019},
  month_numeric = {7}