Vision Based Surgical Tool Tracking and Force Estimation with Robot Kinematics Prior

Thesis (Ph.D.)--University of Washington, 2020 ; Robot assisted minimally invasive surgery combines the skill and techniques of highly-trained surgeons with the robustness and precision of machines. Through a teleoperation scheme, surgeons can execute high-level surgical tasks by commanding instruments controlled by precise robotic devices. Several advantages arise. To name a few: (1) achieved precision is beyond that of human dexterity alone (2) a greater number of kinematic degrees of freedom are possible at the surgical tool tip (3) surgeons are able to operate remotely, i.e. agnostic of patient location given a suitable communication line. Despite the numerous advantages over traditional key-hole or laparoscopic surgery, the lack of realistic and real-time force feedback is a major drawback --- discerning tool-tissue interactions can be unintuitive and can ultimately result in unintentional tissue damage. Directly sensing forces at the tool-tissue interface is theoretically possible using tool tip mounted force sensors, but this approach is not amenable to required sterilization procedures. Thus, a vision based force estimation method is proposed to infer the applied force based on real-time analysis of tissue deformation.