A miniature low-power sensor system for real time 2D visual tracking of LED markers

Humans effortlessly estimate positions of nearby objects in real time purely based on visual perception; a capability that is desirable for many real world robotic scenarios, such as a mobile robot approaching a target, or a robot arm reaching for human-placed objects. Today, such vision based object tracking requires significant computational efforts even for clearly marked objects, because of challenges in real time processing of huge amounts of - mainly redundant - image data (e.g. handling unknown illuminations, disentangling objects from cluttered background). Small autonomous robots typically cannot provide sufficient on-board processing power for visual object tracking. This paper presents a biologically inspired miniature sensor system for real time visual object tracking at rates of several 100 Hz, while utilizing only minimal computing resources. The system combines two functionally separate components: (I) a recently developed Dynamic Vision Sensor Chip (DVS), which - instead of transmitting full image frames at fixed time intervals - asynchronously emits “spike events” that are caused by temporal changes of illumination at individual pixels. Such biologically inspired information encoding drastically reduces the amount of data to be processed compared to traditional video cameras, and significantly increases time resolution. The other component (II) is a 32bit 64MHz microcontroller with 64KB on-board SRAM, which executes an event-based algorithm to track marked objects (here high-frequency flashing LEDs) in real-time, based on the DVS´ output stream of spike events. The complete miniature sensor system requires less than 200mW power to autonomously track markers in real-time at well above 100Hz update rates, for a cost below 10US$ if produced in large quantities. This paper presents the asynchronous event-based tracking algorithm and evaluates the sensor system´s performance in real world robotics scenarios.