Acoustic sonar and video sensor fusion for landmark detection in an under-ice environment

In this paper, we discuss a low-level sensor fusion method to combine datasets from a forward-looking acoustic sonar sensor and an optical camera mounted on an unmanned underwater vehicle. Landmark detection in an underwater environment with typical imaging sensors is prone to errors. Combining the strengths of multiple sensors through fusion allows more accurate detection of objects and landmarks. Objects detected in the sonar data are mapped directly into the camera image as an area of interest in which to search for landmarks. Calibration and cross-calibration of the sensors is presented along with the application of this sensor fusion algorithm on real-world data collected in an under-ice environment. A potential application for this type of sensor fusion is landmark detection and matching using point-features or histogram of gradients methods. Initial application of these landmark matching methods presented here show promise for increased performance of sensor-fusion methods over monocular camera only methods.