Haptic, visual and visuo-haptic softness judgments for objects with deformable surfaces

The purpose of this study is to investigate multisensory visual-haptic softness perception using deformable objects. We created a set of rubber specimens, whose compliance varied in a controlled fashion (0.11 to 0.96 mm/N), but which were otherwise indistinguishable. Participants judged the magnitude of the stimuli according to their softness under haptic-only, vision-only and visuo-haptic conditions. In haptic and visuo-haptic conditions participants explored the stimuli without and with vision of their exploratory movements, respectively. In visual conditions, participants watched how another person explored the stimuli. Participants were well able to differentiate between the different stimuli under all three modality conditions. Stimuli were judged to be slightly softer under vision-only conditions than under haptic-only conditions; visuo-haptic judgments were in-between (average visual weight: 55%). These findings demonstrate that (a) participants can reliably infer softness from indirect visual information alone-that is from watching corresponding exploratory movements and stimulus deformations-, and that (b) such visual information has a major contribution to visuo-haptic softness judgments. We further observed that judgments were more variable under visual as compared to haptic conditions; the variability of visuo-haptic judgments was similar to that of haptic ones. The lack of benefit from adding visual to haptic information, and the contrast between the relatively high visual weight in visuo-haptic judgments on the one hand and the low reliability of visual relative to haptic-only information on the other hand, suggest that the integration of visual and haptic judgments was not optimal, but biased towards vision.