Title :
Random Sampling for Continuous Streams with Arbitrary Updates
Author :
Tao, Yufei ; Lian, Xiang ; Papadias, Dimitris ; Hadjieleftheriou, Marios
Author_Institution :
Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong
Abstract :
The existing random sampling methods have at least one of the following disadvantages: they 1) are applicable only to certain update patterns, 2) entail large space overhead, or 3) incur prohibitive maintenance cost. These drawbacks prevent their effective application in stream environments (where a relation is updated by a large volume of insertions and deletions that may arrive in any order), despite the considerable success of random sampling in conventional databases. Motivated by this, we develop several fully dynamic algorithms for obtaining random samples from individual relations, and from the join result of two tables. Our solutions can handle any update pattern with small space and computational overhead. We also present an in-depth analysis that provides valuable insight into the characteristics of alternative sampling strategies and leads to precision guarantees. Extensive experiments validate our theoretical findings and demonstrate the efficiency of our techniques in practice
Keywords :
database management systems; estimation theory; sampling methods; continuous stream; conventional database; in-depth analysis; random sampling method; Costs; Databases; Heuristic algorithms; Histograms; History; Intrusion detection; Sampling methods; Sampling; selectivity estimation.;
Journal_Title :
Knowledge and Data Engineering, IEEE Transactions on
DOI :
10.1109/TKDE.2007.250588