LAD: Layered Adaptive Data Integration Architecture

Web based data integration systems run in an unpredictable and dynamic environment, which makes the traditional query processing approach inapplicable. Problems necessitating AQP (Adaptive Query Processing) techniques include: (1) statistics may be insufficient; (2) statistics may be imprecise; (3) unavailable data sources may make query results incomplete. In this paper, we present a novel data integration architecture LAD (Layered Adaptive Data integration architecture) to resolve problems mentioned above. Adopted adaptive techniques include: (1) defer making the initial plan if there are not enough high quality statistics; (2) perform re-optimizing operation if distinct deflection is detected; (3) exploit data redundancy to deal with unavailable data sources. Experiment results show that LAD provides satisfactory adaptability and efficiency in the face of uncertainty and dynamics.