Integration of the Senses of Vision and Touch in Perceiving Object Softness

Based upon virtual reality (VR) technologies, a challenging issue for surgical planning is to permit surgeons intuitive and accurate interaction using their sense of vision and touch (e.g. to distinguish the softness of tissues). Since a viewing angle (VA) can influence apparent visual deformation of objects, we hypothesize that the VA would affect the perception of object softness. We conducted an experiment to test this hypothesis and to investigate the mechanisms of integrating the senses of vision and touch. Using a desktop VR setup, we tested 15 human participants for perceiving object softness under 3 conditions: (a) both visual and touch (haptic) information available; (b) only haptic information available; and (c) only visual information available. In each trial, participants had to select the harder object among two deformable balls of same softness but placed in different VAs. Our results showed that the VA affected the perception of object softness (within-subject ANOVA; F = 8.62, p < 0.001) -the larger the VA was the harder the ball was perceived. When two VAs differed at least 15deg, we found a significant difference in perceiving object softness (post-hoc Tukey test, p < 0.05). We applied the method of maximum likelihood estimate to compute the individual and combined weights of visual and haptic information during perceiving object softness. The computation revealed that the visual information was predominant when the VA was at -15deg, whereas the haptic information was prevailing when the VA was +15deg. We also discovered that the variance of both visual and haptic information lay between the individual variances of visual and haptic information, indicating the dependency between visual and haptic information. In conclusion, the VA should never be greater than 15deg to eliminate perceptual illusions. The visual and haptic information depends upon each other, disapproving the assumption of independency in early studies.