Detecting symbols on road surface for mapping and localization using OCR

In this paper, we present a system to detect symbols on roads (e.g. arrows, speed limits, bus lanes and other pictograms) with a common monoscopic or stereoscopic camera system. No manual labeling of images is necessary since the exact definitions of the symbols in the legal instructions for road paintings are used. With those vector graphics an Optical Character Recognition (OCR) System is trained. If only a monoscopic camera is used, the vanishing point is estimated and an inverse perspective transformation is applied to obtain a distortion free top-view. In case of the stereoscopic camera setup, the 3D reconstruction is projected to a ground plane. TESSERACT, a common OCR system is used to classify the symbols. If odometry or position information is available, a spatial filtering and mapping is possible. The obtained information can be used on one side to improve localization, on the other side to provide further information for planning or generation of planning maps.