MapReduce based query of structural engineering experimental data

Structural Engineering Cloud Platform (SECP) is a resource sharing platform oriented to structural engineering domain, and the share of structural engineering experimental data is one of its main functions. In SECP, there are two kinds of experimental data, which are Experimental Description Data (EDD) and Experimental Result Data (ERD). The model of EDD was built based on OWL ontology for shielding the heterogeneity of experimental data from different experimental sites. With the increasing of data scale, the query of EDD has been a bottleneck of the whole management system in SECP. Based on MapReduce, this paper designed and implemented a EDD query system (EDDQS). EDDQS supports query based on SPARQL and there are four core modules named Data Splitter, Query Generator, Job Generator as well as Job Executor. Furthermore, the algorithm of job generation from SPARQL query and job execution algorithms are presented. Finally, the results of our experiments reveal that our MapReduce based query system can handle large amount of EDD efficiently.