Finding the Maximizers of the Information Divergence From an Exponential Family

This paper investigates maximizers of the information divergence from an exponential family ε. It is shown that the rI -projection of a maximizer P to ε is a convex combination of P and a probability measure P- with disjoint support and the same value of the sufficient statistics A. This observation can be used to transform the original problem of maximizing D(·∥ε) over the set of all probability measures into the maximization of a function D̅_r over a convex subset of ker A. The global maximizers of both problems correspond to each other. Furthermore, finding all local maximizers of D̅_r yields all local maximizers of D(·∥E). This paper also proposes two algorithms to find the maximizers of D̅_r and applies them to two examples, where the maximizers of D(·∥ε) were not known before.