• DocumentCode
    1101420
  • Title

    A Novel Framework for Imputation of Missing Values in Databases

  • Author

    Farhangfar, Alireza ; Kurgan, Lukasz A. ; Pedrycz, Witold

  • Author_Institution
    Alberta Univ., Edmonton
  • Volume
    37
  • Issue
    5
  • fYear
    2007
  • Firstpage
    692
  • Lastpage
    709
  • Abstract
    Many of the industrial and research databases are plagued by the problem of missing values. Some evident examples include databases associated with instrument maintenance, medical applications, and surveys. One of the common ways to cope with missing values is to complete their imputation (filling in). Given the rapid growth of sizes of databases, it becomes imperative to come up with a new imputation methodology along with efficient algorithms. The main objective of this paper is to develop a unified framework supporting a host of imputation methods. In the development of this framework, we require that its usage should (on average) lead to the significant improvement of accuracy of imputation while maintaining the same asymptotic computational complexity of the individual methods. Our intent is to provide a comprehensive review of the representative imputation techniques. It is noticeable that the use of the framework in the case of a low-quality single-imputation method has resulted in the imputation accuracy that is comparable to the one achieved when dealing with some other advanced imputation techniques. We also demonstrate, both theoretically and experimentally, that the application of the proposed framework leads to a linear computational complexity and, therefore, does not affect the asymptotic complexity of the associated imputation method.
  • Keywords
    data integrity; database management systems; asymptotic computational complexity; imputation accuracy; imputation methodology; industrial databases; instrument maintenance; linear computational complexity; missing values imputation; research databases; Biomedical engineering; Biomedical equipment; Computational complexity; Councils; Data analysis; Databases; Filling; Instruments; Medical services; Testing; Accuracy; databases; missing values; multiple imputation (MI); single imputation;
  • fLanguage
    English
  • Journal_Title
    Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on
  • Publisher
    ieee
  • ISSN
    1083-4427
  • Type

    jour

  • DOI
    10.1109/TSMCA.2007.902631
  • Filename
    4292217