A new structure-preserving dimensionality reduction approach and OI-net implementation

A new generic nonlinear feature extraction map f is presented based on concepts from approximation theory. Let f map an input data vector x∈ℜⁿ, where n is high, to an appropriate feature vector y∈ℜ^m, where m is sufficiently low. Also let X={X₁,…,X_N} denote an available training set in ℜⁿ. In this paper f is derived by requiring that the geometric structure (metric space attributes) of the points f(X)={f(X₁),…,f(X_N)} in the feature space ℜ^m be as similar as possible to the structure of the points X in the data space ℜⁿ. This is accomplished by selecting first an appropriate dimension m for the feature space ℜ^m according to the size N of the available training set X, subject to bounds on the distortion of the data structure caused by f and on the error in the estimation of the underlying likelihood functions in the feature space. The map f(i) is designed by a multi-dimensional scaling (MDS) approach that minimizes the Sammon´s cost function. This approach uses graph-theoretic (minimal spanning tree) and genetic algorithmic concepts to search efficiently the optimal structure-preserving point-to-point mapping of the training samples X ₁,…,X_N to their images in ℜ^m,Y₁,…,Y_N. Finally, an optimal interpolating (OI) artificial neural network is used to recover the entire function f:ℜ→ℜ^m by interpolating the values y_i=1,…,N at X_i, i=1,…,N. Preliminary simulation results based on this approach are also given