Real-time in-situ visual feedback of task performance in mixed environments for learning joint psychomotor-cognitive tasks

This paper proposes an approach to mixed environment training of manual tasks requiring concurrent use of psychomotor and cognitive skills. To train concurrent use of both skill sets, the learner is provided real-time generated, in-situ presented visual feedback of her performance. This feedback provides reinforcement and correction of psychomotor skills concurrently with guidance in developing cognitive models of the task. The general approach is presented: 1) Sensors placed in the physical environment detect in real-time a learner´s manipulation of physical objects. 2) Sensor data is input to models of task performance which output quantitative measures of the learner´s performance. 3) Pre-defined rules are applied to transform the learner´s performance data into visual feedback presented in realtime and in-situ with the physical objects being manipulated. With guidance from medical education experts, we have applied this approach to a mixed environment for learning clinical breast exams (CBEs). CBE belongs to a class of tasks that require learning multiple cognitive elements and task-specific psychomotor skills. Traditional approaches to learning CBEs and other joint psychomotor-cognitive tasks rely on extensive one-onone training with an expert providing subjective feedback. By integrating real-time visual feedback of learners´ quantitatively measured CBE performance, a mixed environment for learning CBEs provides on-demand learning opportunities with more objective, detailed feedback than available with expert observation. The proposed approach applied to learning CBEs was informally evaluated by four expert medical educators and six novice medical students. This evaluation highlights that receiving real-time in-situ visual feedback of their performance provides students an advantage, over traditional approaches to learning CBEs, in developing correct psychomotor and cognitive skills.