Architecture of Distributed Data Aggregation Service

The ever-growing trend of deploying applications over the Internet has resulted in increasingly tougher constraints and requirements. Data management systems are a major concern when it comes to scalability, flexibility and reliability due to being implemented in a distributed way. In this paper we present a Distributed Data Aggregation Service relying on a storage system designed to meet these demands, namely Blob Seer. The primary goal is to serve as a repository backend for complex analysis and automatic mining of scientific data (like bibtex entries). Several requirements, derived from this objective, match Blob Seer´s features: versioning used for lock-free access to data and different granularity of read / write operations. We proposed a model to perform the correct translation between Blob Seer´s unstructured view of data and the user´s structured view. We implemented a client providing a formal description for the data retrieval queries and a specification for a search API. A benchmark tool relying on a performance model of Blob Seer, will be used to automatically determine the best Blob Seer deployment configuration for a specific data aggregation pattern.