Catadioptric omnidirectional vision sensor integration for vehicle-based sensing

This paper presents a new imaging sensor that combines lenses and mirrors (catadioptric sensors) with a common optical axis to perceive different information around a vehicle. The imaging sensor consists of a central view catadioptric system that returns a panoramic view of the surrounding area using a hyperbolic mirror and a non-central catadioptric system that returns a wide perspective view of the ground plane. Together, these catadioptric sensors provide over 70% of the spherical view, with two image views around the vehicle. By configuring the sensors to be co-axial with a displacement along the optical axis, the sensors retain their field of view with minimal occlusion. This new imaging sensor has applications in driver assistance systems and automated vehicles where the visual information can be used for vehicle classification, lane detection, as well as stereo vision. In this paper, we present the sensor setup, the calibration method as well as an experimental scenario for a driver assistance system that recovers depth for a sparse set of distinct features in both views.