Visualization of turn-taking and mental workload in collaborative working environment

Critical situations such as running into an impasse and failure of decision-making during collaborative work are often hard to recognize by team members themselves. Such critical situations drive the collaborative work into poor continuity. Remote collaboration is even harder to stay active due to communication difficulty in decision-making. This study argues an effective visualization method for monitoring causal factors of conversational situation by multimodal dialogue analysis from turn-taking and mental workload for collaborative working and communication use. Our approach to Follow Awareness visualizes frequency of turn-taking and changing mental workload by the analysis of an EEG (electroencephalogram) based index. Mental workloads of two team members are compared as difference in their changes within a predetermined amount of times for the analysis of follows between the team members. Frequency of turn-taking and changes in mental workload are then displayed by a mobile device during their collaborative work for validation in real-time. Evaluation results from a case study of collaborative work in pair programming presented large difference of changes in mental workload at the point their active communication turned to silence by a small setback.