Feature selection based on Mutual Information in supervised learning

How to use finite samples to effectively and efficiently estimate high-dimension MI(Mutual Information) is a crucial problem in MI based feature selection. In this paper, we propose a novel method of estimating high-dimension MI using clustering and corresponding algorithm applied in feature selection. This method is different from many proposed methods by others, which the high-dimension MI are not estimated directly. Theoretical analysis and practical evaluation of our algorithm are also included in this paper.