Real-time 3D image reconstruction guidance in liver resection surgery

Background: Minimally invasive surgery represents one of the main evolutions of surgical techniques.However, minimally invasive surgery adds difficulty that can be reduced through computer technology.Methods: From a patient’s medical image [US, computed tomography (CT) or MRI], we have developedan Augmented Reality (AR) system that increases the surgeon’s intraoperative vision by providing a virtualtransparency of the patient. AR is based on two major processes: 3D modeling and visualization of anatomicalor pathological structures appearing in the medical image, and the registration of this visualization ontothe real patient. We have thus developed a new online service, named Visible Patient, providing efficient3D modeling of patients. We have then developed several 3D visualization and surgical planning softwaretools to combine direct volume rendering and surface rendering. Finally, we have developed two registrationtechniques, one interactive and one automatic providing intraoperative augmented reality view.Results: From January 2009 to June 2013, 769 clinical cases have been modeled by the Visible Patientservice. Moreover, three clinical validations have been realized demonstrating the accuracy of 3D models andtheir great benefit, potentially increasing surgical eligibility in liver surgery (20% of cases). From these 3Dmodels, more than 50 interactive AR-assisted surgical procedures have been realized illustrating the potentialclinical benefit of such assistance to gain safety, but also current limits that automatic augmented reality willovercome.Conclusions: Virtual patient modeling should be mandatory for certain interventions that have now to bedefined, such as liver surgery. Augmented reality is clearly the next step of the new surgical instrumentationbut remains currently limited due to the complexity of organ deformations during surgery. Intraoperativemedical imaging used in new generation of automated augmented reality should solve this issue thanks to thedevelopment of Hybrid OR.