Comparing and Combining Predictive Business Process Monitoring Techniques

Predictive business process monitoring aims at forecasting potential problems during process execution before they occur so that these problems can be handled proactively. Several predictive monitoring techniques have been proposed in the past. However, so far those prediction techniques have been assessed only independently from each other, making it hard to reliably compare their applicability and accuracy. We empirically analyze and compare three main classes of predictive monitoring techniques, which are based on machine learning, constraint satisfaction, and Quality-of-Service (QoS) aggregation. Based on empirical evidence from an industrial case study in the area of transport and logistics, we assess those techniques with respect to five accuracy indicators. We further determine the dependency of accuracy on the point in time during process execution when a prediction is made in order to determine lead-times for accurate predictions. Our evidence suggests that, given a lead-time of half of the process duration, all predictive monitoring techniques consistently provide an accuracy of at least 70%. Yet, it also becomes evident that the techniques differ in terms of how accurately they may predict violations and nonviolations. To improve the prediction process, we thus exploit the characteristics of the individual techniques and propose their combination. Based on our case study data, evidence indicates that certain combinations of techniques may outperform individual techniques with respect to specific accuracy indicators. Combining constraint satisfaction with QoS aggregation, for instance, improves precision by 14%; combining machine learning with constraint satisfaction shows an improvement in recall by 23%.