Large-Scale Dependency Knowledge Acquisition and its Extrinsic Evaluation Through Word Sense Disambiguation

Knowledge plays a central role in intelligent systems. Manual knowledge acquisition is very inefficient and expensive. In this paper, we present (1) an automatic method to acquire a large amount of lexical-dependency knowledge, and (2) an innovative knowledge representation model to effectively minimize the impact of noise and improve knowledge quality. We also propose a new type of knowledge base evaluation-extrinsic evaluation, which evaluates knowledge by its impact to an external application. In our experiments we adopt word sense disambiguation (WSD) as the extrinsic evaluation measure. Due to the lack of sufficient knowledge, existing WSD methods either are brittle and only capable of processing a limited number of topics or words, or provide only mediocre performance in real-world settings. With the support of acquired knowledge, our unsupervised WSD system significantly outperformed the best unsupervised systems participating in SemEval 2007, and achieved the disambiguation accuracy approaching top-performing supervised systems.