A method of 3D object reconstruction by fusing vision with touch using internal models with global and local deformations

A method for sensor fusion of vision and touch using an internal model with both global and local deformation is introduced. Superquadratics with local deformations are used as internal models. The proposed method consists of two phases. In the first phase, the object shape is recovered parametrically by changing the parameters of superquadrics using visual data. The internal model constructed in this phase is a rough representation of object shape. The use of this parametric model, permits easy extraction of regions which are invisible and/or have large errors. In the second phase, the tactile sensor is made to explore in order to obtain information of the above-mentioned regions, and to deform the internal model locally based on the defined energy functions. The feasibility of the proposed method is confirmed by simulations