Augmented reality for complex surgical procedures

Abstract

[EN] In 2014, 7 million of CT and MRI scans were used pre and intraoperatively in the UK, flat 2D images whose potential is not fully used. Surgeons can use Volume rendering techniques to create a 3D model which they will display in a monitor; however, an isolation between Virtual and Real world can be observed. During an MSc Group Project, an Augmented Reality System was developed to integrate a Head Mounted Display together with optical cameras that track surgeon’s position of the eyes and therefore allows to view the 3D model directly through these goggles. However, tracking inaccuracies, lack of registration process, real-time delays and difficulties to focus in the virtual object, made it impossible to bring to a surgery level. The methodology used in this project has been to review specific sources from both Computer Science and healthcare, discover new techniques that can improve the overall AR System and develop a be-spoken AR System that include these techniques. After testing the technology with a cadaver, results show that this research has improved the tracking accuracy by a 95%, allows users to accurately merge Real and Virtual world by providing Active Stereoscopy and provides marker-less patient tracking by using three anatomical points. Furthermore, it provides a larger range of tracking than previous researches thanks to Advanced Real Time optical cameras and proves the importance of Active Stereoscopy for user’s depth perception. Although improvements regarding rotation and scale are still needed; all in all, this research brings Augmented Reality from 4 to 6 Technology Readiness Level.