ABSTRACT

Handling the data sparseness question is a main way to further enhance the system performances of Head-driven statistical parsing models. Two smoothing methods are proposed to mitigate remaining data-sparseness problems. The first smoothing method is that two word classification algorithms based on word similarity have been developed, which employ the mutual information of two words that are adjoining words or have semantic relationship to define word similarity and word-class similarity. The second smoothing method is to decompose the generation of each internal rule into a sequence of smaller steps, and then to make conditional independence assumptions to incorporate the Part-Of-Speech tags of adjoining words or adjoining phrase tags into the probability computation of the context-free rules, the incorporating additional context information into the syntactic parsing models is very useful for improving the system performances of syntactic parsing. The two category-based statistical analysis models are tested through experiments. The improved parsing model 2 has far better system performances than head-driven parsing model: recall reaches 87.89%, accuracy reaches 88.62, and F-measure is enhanced 8.10% compared with the head-driven analysis method.

摘要

处理数据稀疏性问题是进一步提高Head驱动的统计分析模型的系统性能的主要方法。提出了两种平滑方法来缓解剩余的数据稀疏性问题。第一种平滑方法是开发了两种基于词相似度的词分类算法，该算法利用两个相邻词或具有语义关系的词的互信息来定义词相似度和词类相似度。第二种平滑方法是将每个内部规则的生成分解为一系列较小的步骤，然后进行条件独立性假设，以将相邻单词或短语短语的词性标签合并到上下文的概率计算中-无规则，将额外的上下文信息合并到语法分析模型中对于提高语法分析的系统性能非常有用。通过实验测试了两个基于类别的统计分析模型。改进的解析模型2的系统性能远优于头部驱动的解析模型：与头部驱动的分析方法相比，召回率达到87.89％，准确性达到88.62，F度量提高了8.10％。