The power of the dark side: using cast shadows for visually-guided touching

A humanoid robot needs to be able to operate safely in human workspaces, preferably without extensive calibration. We consider the problem of directing a robot arm towards, across, and away from an unmodeled surface without damaging it. For this task we make use of a powerful resource: the shadow cast by the robot´s own body. We show that the cast shadow of the arm on the surface can be detected by a camera and used to derive a time-to-contact estimate. This estimate, when combined with the 2D tracked location of the arm´s endpoint in the camera image, is sufficient to allow 3D control relative to the surface. We show that the same method can detect either cast shadows or reflections, allowing the robot to operate correctly over water, mirrors, or other reflective materials. Such scenarios, along with low-texture surfaces, are cases in which stereovision - the more commonly used depth cue in robotics - might fail and is worth augmenting. We draw on the literature of computer graphics and virtual reality to argue that for manipulation, sensitivity to shadows and interreflection will be of similar importance to stereovision as a depth cue when attempting to touch an object. In computer vision, shadows are generally treated as a nuisance, but that doesn´t mean roboticists should do the same.