Performance comparison framework for energy disaggregation systems

Energy disaggregation algorithms decompose building-level energy data into device-level information. We conduct a head-to-head comparison of energy disaggregation techniques across multiple metrics and data sets. Our framework for analyzing the performance of a complete energy disaggregation system includes event detection, classification, and power assignment. We use receiver operating characteristics (ROCs) to evaluate event detection performance, and we introduce a technique to evaluate device-level event detection. We use confusion matrices to compare classification performance across several classifiers, and evaluate the resulting power assignments using several assignment metrics that are commonly used in the literature to demonstrate the varying strengths of the techniques that were considered. We apply this framework to several publicly available datasets and demonstrate how system performance varies with sampling frequency and the inclusion of reactive power. Our results suggest that (1) disaggregation performance varies considerably across data sets (2) increased data sampling rate improves disaggregation performance, and (3) additional features such as reactive power yields disaggregation performance improvements.