Learning features with structure-adapting multi-view exponential family harmoniums

Existing multi-view feature extraction methods are based on restrictive assumptions on the connections between feature vectors and input data. These assumptions damage the quality of learned features, and also require more effort on choosing right dimensions of feature vector components connected to each view. In this paper we present adaptive multi-view harmonium (SA-MVH) for multi-view feature extraction, where its each hidden node chooses the views to connect with while training phase via switch parameters. "Switch" parameters are multiplied to the connection weights of ordinary exponential family harmoniums (EFH) to decide the existence of connection between hidden nodes and views. With switch parameters, a SA-MVH automatically adapts its structure to achieve better representation of data distribution. The model can also be easily trained using the same training algorithms used for EFHs. Numerical experiments on synthetic and real-world datasets demonstrate the useful behavior of the SA-MVH, compared to the existing multi-view feature extraction methods.