Reducing the Cluster Monitoring Workload by Identifying Application Characteristics

Monitoring is crucial for effective management and efficient utilization of the cluster computers. The information extracted from the node by the monitoring tools is of different volume and accuracy with different monitoring purposes. The overhead of monitoring will increase with the increase of monitoring tasks. Also large volume of data needs to be managed and transferred to the monitoring application system. In this paper, we present an approach for reducing the monitoring workload by identifying the main characteristics of the application. The main characteristics called main factors are identified by performing principal component analysis (PCA) on the fly of application execution. Upon identifying main factors, we further category them into common factors and specific factors. A strategy for improving the efficiency of monitoring using the knowledge of application characteristics is proposed. A prototype monitoring system adopting this strategy is implemented. Experiments with a couple of typical benchmarks have been conducted to validate our approach. The results show that our approach is effective and improves efficiency and availability of the monitoring system.