DocumentCode :
109956
Title :
Minimum KL-Divergence on Complements of L_{1} Balls
Author :
Berend, Daniel ; Harremoes, Peter ; Kontorovich, Aryeh
Author_Institution :
Dept. of Math. & Comput. Sci., Ben-Gurion Univ., Beer-Sheva, Israel
Volume :
60
Issue :
6
fYear :
2014
fDate :
Jun-14
Firstpage :
3172
Lastpage :
3177
Abstract :
Pinsker´s widely used inequality upper-bounds the total variation distance ∥P - Q∥1 in terms of the Kullback-Leibler divergence D(P∥Q). Although, in general, a bound in the reverse direction is impossible, in many applications the quantity of interest is actually D*(v, Q)-defined, for an arbitrary fixed Q, as the infimum of D(P∥Q) over all distributions P that are at least v-far away from Q in total variation. We show that D*(v, Q) ≤ Cv2 + O(v3), where C = C(Q) = 1/2 for balanced distributions, thereby providing a kind of reverse Pinsker inequality. Some of the structural results obtained in the course of the proof may be of independent interest. An application to large deviations is given.
Keywords :
information theory; probability; KL-divergence; Kullback-Leibler divergence; reverse Pinsker inequality; total variation distance; Abstracts; Atomic measurements; Equations; Extraterrestrial measurements; History; Information theory; Standards; McDiarmid´s inequality; Pinsker´s inequality; Sanov´s theorem;
fLanguage :
English
Journal_Title :
Information Theory, IEEE Transactions on
Publisher :
ieee
ISSN :
0018-9448
Type :
jour
DOI :
10.1109/TIT.2014.2301446
Filename :
6746175
Link To Document :
بازگشت