Different models for predicting driving performance in people with brain disorders

Data from performance on a computerized battery of driving-related sensory-motor and cognitive tests (SMCTests™) were used to predict outcome on a blinded on-road driving assessment in 501 people with brain disorders. Six modelling approaches were assessed: discriminant analysis (DA), binary logistic regression (BLR), nonlinear causal resource analysis (NCRA), and three kernel methods (product kernel density (PK), kernel-product density (KP), and support vector machine (SVM)). At the classification level, the three kernel methods were more accurate for predicting on-road Pass or Fail (SVM 99%, PK 99%, KP 80%) than the other models (DA 75%, BLR 77%, NCRA 66%). However, accuracy decreased substantially across the kernel models when leave-one-out cross-validation was used to estimate how accurately the models would predict on-road Pass or Fail in an independent referral group (SVM 76%, PK 73%, KP 72%) but remained fairly constant for DA (74%) and BLR (76%). Cross-validation of NCRA was not possible. While kernel-based models are successful at modelling complex data at a classification level, this appears to be due to overfitting of the data which does not improve accuracy in an independent data set over and above the accuracy of other modelling techniques.