Title :
Quickest search for a rare distribution
Author :
Malloy, Matthew L. ; Tang, Gongguo ; Nowak, Robert D.
Author_Institution :
Electr. & Comput. Eng., Univ. of Wisconsin-Madison, Madison, WI, USA
Abstract :
We consider the problem of finding one sequence of independent random variables following a rare atypical distribution, P1, amongst a large number of sequences following some null distribution, P0. We quantify the number of samples needed to correctly identify one atypical sequence as the atypical sequences become increasingly rare. We show that the known optimal procedure, which consists of a series of sequential probability ratio tests, succeeds with high probability provided the number of samples grows at a rate equal to a constant times π-1 D(P1∥P0)-1, where π is the prior probability of a sequence being atypical, and D(P1∥P0) is the Kullback-Leibler divergence. Using techniques from sequential analysis, we show that if the number of samples grow at a rate equal to π-1 D(P1∥P0)-1, any procedure fails. This is then compared to sequential thresholding [1], a simple procedure which can be implemented without exact knowledge of distribution P1. We also show that the SPRT and sequential thresholding are fairly robust to our knowledge of π. Lastly, a lower bound for non-sequential procedures is derived for comparison.
Keywords :
statistical distributions; SPRT; independent random variables; null distribution; quickest search; rare atypical distribution; sequential analysis; sequential probability ratio tests; sequential thresholding; Computers; Educational institutions; Electronic mail; Indexes; Nickel; Search problems; Testing; CUSUM procedure; Rare events; SPRT; sequential analysis; sequential thresholding; sparse recovery; spectrum sensing;
Conference_Titel :
Information Sciences and Systems (CISS), 2012 46th Annual Conference on
Conference_Location :
Princeton, NJ
Print_ISBN :
978-1-4673-3139-5
Electronic_ISBN :
978-1-4673-3138-8
DOI :
10.1109/CISS.2012.6310773