Test and evaluation for determining screener training effectiveness

For many years, the only FAA-approved checkpoint screener-training program was a classroom lecture developed by the Air Transport Association. In April 1997, the FAA also approved the use of a Computer-Based Training (CBT) program for initial screener training prior to on-the-job training. This study describes a comprehensive test and evaluation of four training programs currently in use at US airports. The evaluation will determine the efficacy of the individual training programs. Because the variety of training options is growing, the FAA has developed a single uniform measure of mastery of initial training, the Screener Readiness Test (SRT). This preparedness evaluation is designed to test newly trained screeners for checkpoint related knowledge and abilities. The test contains multiple-choice questions on checkpoint rules and the major checkpoint screening tasks. It also measures screeners´ ability to detect threat articles in X-ray images of bags. One hundred ninety two new security screeners participated in this study. Each screener was randomly assigned to one of the four training systems (i.e., 48 screeners per training system). Half of these screeners were tested with SRT immediately after training. This measured their screener-relevant knowledge and abilities. The results allowed us to determine which training system produced the most capable screeners, thus allowing us to compare the four training systems. The other half of the screeners took the SRT as a pretest prior to training and then again after training. This pretest/posttest within-subjects design allowed us to measure the amount of learning that could be attributed to each training system. An additional overall performance measure was developed taking into. account screener candidate dropout rates. A multiplicative model was used in which either, low SRT test scores or high attrition rates would lead to a poor overall performance score.. The paper discusses the outcome of this effort and report on the testing and performance of the classroom and. CBT training systems using the SRT