Assessing Design of Online Courses Using Bayesian Belief Networks

Even with recent research showing students enrolled in online courses are outperforming those in traditional courses, the need to assess the instructional design of online courses is greater than ever due to increasing enrollments and attrition rates that continue to be significantly higher than traditional courses. Most past research used surveys for course assessment, which measure only perceptions. Recently researchers have proposed more holistic assessments of online courses using Bayesian Belief Networks (BBNs) to integrate multiple assessment instruments. While the advantages of BBNs for educational assessments have been demonstrated, past research used only simulated data. This study furthers past research by testing the suitability of BBN-based methodologies for real-time assessment of online courses using larger data sets with missing and sparse data. Testing used actual course data from two undergraduate online Java programming courses. Past literature identified the creation of the conditional probability tables (CPTs) as the greatest challenge of using BBNs. This research also investigated the automatic population of CPTs via software. Tests revealed excellent performance with large data sets demonstrating the scalability of BBN-based assessment methodologies for online courses with large enrollments. With a well-designed network, most of the CPTs can be populated via software, significantly reducing the time and effort required to design and use the BBN. A number of recommendations to improve usability, performance, and accuracy of BBN-based course assessment methodologies are demonstrated. Future research should provide a comparative analysis between BBNs and other methodologies for the assessment of online course design.