On the fidelity of human skill models

Modeling dynamic human control strategy, or human skill, in response to real-time sensing is becoming an increasingly popular paradigm in many research areas. These models are learned from experimental data, and as such can be characterized despite the lack of a good physical model. Unfortunately, learned models presently offer few, if any, guarantees in terms of model fidelity to the source data. As such, we propose an independent, post-training model validation procedure based on hidden Markov models (HMMs). The proposed method generates a stochastic similarity measure comparing system trajectories for the source process and the learned models. Using this method, we are able to verify model fidelity. We demonstrate the proposed method in the validation of neural-network models for real-time human driving skill