Approximation theory of output statistics

Given a channel and an input process with output statistics that approximate the original output statistics with arbitrary accuracy, the randomness of the input processes is studied. The notion of resolvability of a channel, defined as the number of random bits required per channel use in order to generate an input that achieves arbitrarily accurate approximation of the output statistics for any given input process, is introduced. A general formula for resolvability that holds regardless of the channel memory structure is obtained. It is shown that for most channels, resolvability is equal to the Shannon capacity. By-products of the analysis are a general formula for the minimum achievable source coding rate of any finite-alphabet source and a strong converse of the identification coding theorem, which holds for any channel that satisfies the strong converse of the channel coding theorem