Independent component regression based on mutual information maximization

A new independent component regression (ICR) algorithm which maximizes the mutual information (MI) between extracted latent variables (LV) and output variable is proposed. It is found that mutual information between extracted LVs and output variable can be delicately combined with the independent component analysis (ICA) objective and only two-dimensional joint entropy needs to be estimated, which can be approximated by Edgeworth expansion. Balance is achieved between maximizing statistical independency and mutual information by forming a dual objective optimization problem. The performance of the proposed algorithm is tested on both simulation examples and real data sets.