• DocumentCode
    170393
  • Title

    The modifications of the LRV algorithm in a new method of word segmentation ESA

  • Author

    Hanshi Wang ; Qiujie Fu ; Lizhen Liu ; Wei Song ; Jingli Lu

  • Author_Institution
    Inf. & Eng. Coll, Capital Normal Univ., Beijing, China
  • fYear
    2014
  • fDate
    16-18 May 2014
  • Firstpage
    163
  • Lastpage
    167
  • Abstract
    This article proposes two modifications of a new unsupervised method of word segmentation consisting of three phases: Evaluation, Selection, and Adjustment (ESA), which was presented in our early paper. Lowest Relative Value (LRV) is the core algorithm in ESA The whole method has only one parameter (the exponent in LRV) that can be approximately predicted by the empirical formulae. In this article, we further evaluate one of the empirical formulae on SIGHAN Bakeoff-3 dataset. And we correct the formula for the better prediction. Additionally, we modify another part of the LRV and demonstrate how the part alleviates the sparse data problem. Meanwhile, the results indicate that the modified ESA can produce better results than the original ESA and other methods.
  • Keywords
    learning (artificial intelligence); word processing; ESA; LRV algorithm; SIGHAN Bakeoff-3 dataset; evaluation selection adjustment; lowest relative value; word segmentation; Computational linguistics; Conferences; Entropy; Regression analysis; Training; Tuning; Uncertainty; ESA; data sparseness; parameter prediction; unsupervised; word segmentation;
  • fLanguage
    English
  • Publisher
    ieee
  • Conference_Titel
    Progress in Informatics and Computing (PIC), 2014 International Conference on
  • Conference_Location
    Shanghai
  • Print_ISBN
    978-1-4799-2033-4
  • Type

    conf

  • DOI
    10.1109/PIC.2014.6972317
  • Filename
    6972317