Toward a subjective synchronous communication in multimodal human-machine interaction: Intention of movement alternates simultaneous perception in auditory-tactile temporal order judgment

We investigated the effect of voluntary movement on temporal multimodal integration using a psychological experiment. The present study used auditory-tactile Temporal Order Judgment (TOJ) task to measure the differences in Point of Subjective Simultaneity (PSS) and Just Noticeable Difference (JND). To distinguish the effect of voluntary movement and proprioceptive feedback, we prepared three conditions: Voluntary condition, Involuntary condition, and No-movement condition. Voluntary movement and Involuntary movement shifted PSS to the point where an auditory stimulus presented prior to a tactile stimulus compared to No-movement condition. Furthermore, the shift of PSS in Voluntary condition was larger than that in Involuntary condition. JND of Voluntary condition was smaller than those of the other two conditions. These results revealed that voluntary movement improved temporal resolution between auditory and tactile stimuli. Moreover, it was demonstrated that voluntary movement also alternates audio-tactile temporal integration.