A real-time vision-based method to overlay medical imaging data onto a moving patient

This paper will present a novel movement invariant approach for overlaying surgical navigational data on to a live, and moving patient. A pico projector class image projector was used to project an x-ray onto a human forearm, which was sensed using a novel computer vision algorithm. The computer vision algorithm segmented the forearm and found a vector of surrounding points that was then tested for differing forearm thicknesses. Once the forearm was located, an x-ray image was transformed and projected. The projected x-ray was accurate to within 7-10 pixels or 14 to 20mm at a rate of 0.73 seconds per iteration using python and Opencv on an Intel generation 4 i7 laptop. The results could be improved by a projector with a shorter focal length and larger field of view so that the forearm could be much closer to the camera.