Cleaning Framework for Big Data - Object Identification and Linkage

Data is a valuable resource. The proper use of high-quality data can help make better predictions, analysis and decisions. Poor-quality data is detrimental to data analytics. Data from different sources may provide the same entities, but different identities. This becomes a concern particularly when large-scale heterogeneous data from multiple sources are integrated for other purposes. This paper aims to identify same or similar objects and link these associated objects together so that the data can be cleaned and combined efficiently. Our research harnesses both context and usage patterns of data items to determine relationships among objects. Our experimental results show that efficient linkage among multiple sources can be constructed using context and usage patterns.