A novel adaptive fuzzy load balancer for heterogeneous LAM/MPI clusters applied to evolutionary learning in neuro-fuzzy systems

Load balancing in parallel master-slave implementations on heterogeneous computing clusters is a pressing research problem. Proper load balancing can lead to dramatic speedups in program run times. This paper introduces a novel adaptive fuzzy load balancer which automatically senses cluster state through measurements of node evaluation times and network delays. Measured data are collected within a time window and then clustered using fuzzy c-means clustering. The optimal number of clusters are decided using the Xie-Beni index. Rule base extraction is facilitated by reverse projection of clusters (for antecedents) and a heuristic function (for consequents). Re-clustering is triggered on outlier point detection, and re-validation of clusters is performed depending on an FCM objective function-based cluster scattering threshold. The load balancer is deployed on the master to balance the load between various slaves. The algorithm is tested extensively on an evolutionary-neuro-fuzzy network learning application and implemented in a LAM/MPI computing environment. Results clearly bring out the efficacy of employing the adaptive load balancer in heterogeneous computing environments. Speedups ranging from 42% to 89% are observed when compared to parallel implementations without the fuzzy load balancer, and up to 448% when compared to the serial implementations.