Enhanced ultrasound imaging resolution with 3D optical patch imagery

This paper presents a method of improving 3D ultrasound image resolution by fusing the 3D ultrasound images with 3D optical imagery obtained from a limited field of view. Organ surfaces are reconstructed using fine samples from the optical image of a single side of the organ and coarse samples from ultrasound image of the hidden side. The reconstruction is based on the fact that all deformations of an organ surface lie in low dimensional structured subspaces that can be learned using training data obtained from different subjects. The multimodality fusion algorithm accounts for differences in resolution and noise levels between the different modalities. Experiment results using ex-vivo MRI, ultrasound and optical images provide the first demonstration of improving ultrasound image resolution by an average of 2.6 mm using optical images. Surface details not seen on the ultrasound image are clearly visible on the reconstructed image. Specifically, the enhanced reconstruction achieves an average error level of less than 3 mm on the hidden organ surface.