Structured web information extraction using repetitive subject pattern

Data records on a dynamic web page are often generated from databases with fixed templates or layouts by server-side scripts. Generally, each data record on the web page has a subject item that can be used to identify a data record. This paper reports a novel semi-supervised information extraction system that lets end-users give only one subject item of sample data record. The system then builds a wrapper and extracts the relevant data records automatically. The techniques for the proposed system are a repetitive subject pattern for discovery data records, a subject tree clustering algorithm for clustering target data records, and a subject tree alignment for aligning data items and create an extraction pattern. For performance evaluation purpose, the proposed system is empirically tested on twelve popular real world websites both Thai and English. It provides the outstanding result by reporting 100 percentage of accuracy for correct extracted records. In addition, the proposed system shows higher degree of being user friendly when compared with other similar systems.