Estimating the multivariate conditional density using relatively sparse training data pairs

Unlike most nonlinear system identification tasks, which focus on determining the unknown system functions parametrically or nonparametrically to be used in future prediction of the output values given specific inputs, many statistical signal processing applications require the estimation of the corresponding conditional density function so that a probabilistic decision can be made. In this paper, we propose a new method to estimate the multivariate conditional density, f(m/x), a density over the output space m conditioned on any given input x. In particular, we are interested in cases where the number of available training data, points is relatively sparse within x space. We start from a priori considerations and establish certain desirable characteristics in kernel functions for conditional density estimation. We find that Gaussian kernels with expanding covariances, expanding as we move away from the data point of the kernel, satisfy these a priori considerations. We combine these expanding Gaussian kernels (EGK) according to Bayesian techniques. We compare the EGK with standard Gaussian kernel (SDK) methods, and find that EGK avoids multimodality, has diminishing confidence levels farther from training points, performs better asymptotically and performs better with respect to the Kullback-Leibler criteria