A nonsingular linear transformation of binary-valued random vectors

which minimizes a mutual information criterion I(y) is considered. It is shown that a nonsingular A exists such that

if and only if

has a generalized binomial distribution. Computational algorithms for seeking an optimal

are developed, and dimensionality reduction is discussed briefly. This linear transformation is useful in improving the approximation of probability distributions. Numerical examples are presented.