Estimation of deformable object properties from shape and force measurements for virtualized reality applications

The paper addresses the topic of intelligent sensing and mapping of deformable objects´ properties for virtualized reality applications. Shape information in form of contours of soft deformable objects tracked over a sequence of images is correlated to the interaction measurements collected at the level of the fingers of a robotic hand by means of neural networks. The proposed solution allows the automated and implicit modeling of the actual elastic behavior without a priori knowledge on the material of an object. It also provides the ability for the application to estimate the shape of an object for unrecorded interactions. Experimental results presented for several soft objects show the ability of the proposed solution to accurately capture and predict severe shape deformations in spite of slight changes in lighting, contrast and background.