An integrated vision/ultrasonic sensor for 3D target recognition and measurement

A system which combines an ultrasonic array system with a vision system may offer the advantages of both while overcoming some of their individual disadvantages. For example, a vision system can not easily get range information from a single view, while the ultrasonic system readily gives this; on the other hand, the angular resolution of the ultrasonic system is quite limited compared with that of the vision system. By integrating the two sensors, 3D targets may be identified and measured and the results may be used to guide a robot to pick up targets or avoid obstacles. To demonstrate the advantages of such an integrated system, a prototype system has been set up which combines the Nottingham ultrasonic array with a vision system; this system is described. The arrangement of the sensors is first described, and the way in which the ultrasonic phased arrays works is then discussed. The method by which the vision system has been calibrated is explained, and the various functions performed by the system are outlined. Finally, the limitations of the current system are considered and possible future developments discussed