Title :
Entropy for singular distributions
Author :
Pichler, G. ; Koliander, Gunther ; Riegler, Erwin ; Hlawatsch, Franz
Author_Institution :
Inst. of Telecommun., Vienna Univ. of Technol., Vienna, Austria
fDate :
June 29 2014-July 4 2014
Abstract :
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables (which are neither discrete nor continuous) has not been available so far. Here, we propose such an extension for the practically relevant class of singular probability measures that are supported on a lower-dimensional subset of Euclidean space. We show that our entropy transforms in a natural manner under Lipschitz functions and that it conveys useful expressions of the mutual information. Potential applications of the proposed entropy definition include capacity calculations for the vector interference channel, compressed sensing in a probabilistic setting, and capacity bounds for block-fading channel models.
Keywords :
channel capacity; compressed sensing; entropy; fading channels; functions; interference; probability; set theory; vectors; Lipschitz functions; block-fading channel models; capacity bounds; capacity calculations; compressed sensing; differential entropy; information theory; lower-dimensional Euclidean space subset; probabilistic setting; singular distributions; singular probability measures; vector interference channel; Probabilistic logic; information entropy; information measures; mutual information; rectifiable sets; singular measures;
Conference_Titel :
Information Theory (ISIT), 2014 IEEE International Symposium on
Conference_Location :
Honolulu, HI
DOI :
10.1109/ISIT.2014.6875281