Header logo is

An Interactive Augmented-Reality Video Training Platform for the da Vinci Surgical System

2017

Miscellaneous

hi


Teleoperated surgical robots such as the Intuitive da Vinci Surgical System facilitate minimally invasive surgeries, which decrease risk to patients. However, these systems can be difficult to learn, and existing training curricula on surgical simulators do not offer students the realistic experience of a full operation. This paper presents an augmented-reality video training platform for the da Vinci that will allow trainees to rehearse any surgery recorded by an expert. While the trainee operates a da Vinci in free space, they see their own instruments overlaid on the expert video. Tools are identified in the source videos via color segmentation and kernelized correlation filter tracking, and their depth is calculated from the da Vinci’s stereoscopic video feed. The user tries to follow the expert’s movements, and if any of their tools venture too far away, the system provides instantaneous visual feedback and pauses to allow the user to correct their motion. The trainee can also rewind the expert video by bringing either da Vinci tool very close to the camera. This combined and augmented video provides the user with an immersive and interactive training experience.

Author(s): Carlson, Jaimie and Kuchenbecker, Katherine J.
Year: 2017
Month: May

Department(s): Haptic Intelligence
Bibtex Type: Miscellaneous (misc)
Paper Type: Workshop

Address: Singapore
How Published: Workshop paper (3 pages) presented at the ICRA Workshop on C4 Surgical Robots
State: Published

BibTex

@misc{Carlson17-ICRAWS-Augmented,
  title = {An Interactive Augmented-Reality Video Training Platform for the da {V}inci Surgical System},
  author = {Carlson, Jaimie and Kuchenbecker, Katherine J.},
  howpublished = {Workshop paper (3 pages) presented at the ICRA Workshop on C4 Surgical Robots},
  address = {Singapore},
  month = may,
  year = {2017},
  doi = {},
  month_numeric = {5}
}