导　　师： 袁方

学科专业： 081203

授予学位： 硕士

作　　者： ;

机构地区： 河北大学

摘　　要： 随着计算机软硬件技术的快速发展,电子化文本数据急剧增加。如何有效利用这些电子化文本数据,已成为人们的研究热点之一。基于人工智能技术的自动文本分类是有效利用电子化文本数据的基础。 本文首先给出了类别核心词的概念,把在文本的标题、摘要和关键词部分出现的特征词称为类别核心词,类别核心词能更好地表示文本的类别特征。然后,基于类别核心词这一概念,从提高文本分类的性能出发,对文本分类技术主要进行了两个方面的研究。 一方面,对朴素贝叶斯分类方法进行了改进,给出了一种基于类别核心词的朴素贝叶斯分类方法。从贝叶斯公式直观来看,文本的类别与特征词的词频有很大关系,词频越大,说明该词越具有代表性,但实际情况并非如此,有些低频特征词可能更能代表文本的类别特征。由于朴素贝叶斯分类是一种基于概率的分类方法,所以,这些低频特征词的概率值可能会近似等于零,这时,小概率事件就发生了。小概率事件发生的情况越多,对文本分类的结果影响也就越大。为了尽量避免这种情况,我们将类别核心词融入到概率计算中,这样,那些属于类别核心词的低频特征词的词频就会得到提高,从而在一定程度上避免了小概率事件的发生。实验结果表明,该方法能有效提高分类准确率。 另一方面,实现了基于概念的文本分类。目前常用的文本分类方法主要是基于特征词进行分类,这不能有效解决同义词和多义词问题,从而影响分类准确率。将文本从特征词空间映射到概念空间,进而在概念空间进行文本分类,可以很好地解决这个问题。本文给出了一种基于类别核心词的概念映射算法,首先对文本中出现的类别核心词进行概念映射,然后再对非类别核心词进行概念映射,最后在概念空间上进行文本分类。实验结果表明,� E-form text data is increasing rapidly along with the fast development of computer software and hardware. As a result, it has become one of the focuses of information techniques on how to utilize this rich data resource. Automatic text categorization, which is based on artificial intelligence technology, is the base of how to effectively utilize e-form text data. We first present a new concept of core words in this paper. A core word can be the representative characteristic appeared in the title, abstract or keywords section of a text. A core word can better represent the characteristic of a text than other keywords. From the perspective of improving the performance of text categorization methods, we make two researches on text categorization techniques, which are based on the concept of core words. Firstly, we improve Naive Bayes text categorization, and present a method of Naive Bayes based on core words. From the formula of Bayesian, the class of a text is related to frequency of keyword, the keywords which appear more frequently would be representative words of this text. But in fact, this is not always true. Some keywords with lower appearance frequency could best represent the class of this text. Because Naive Bayes is a method based on probability, the probability of keywords with lower appearance frequency will be nearly zero. At this time, small probability event will happen. The more the small probability events happen, the more the impact to text categorization results. To avoid this scenario, we add core words into the computation of probability. As a result, the core words with lower appearance frequency will get improved, and the small probability event will get reduced. The experimental results show that, this method-Naive Bayes based on core words-can improve the precision of text categorization effectively. Secondly, we implement a text categorization method based on concept. The common methods of text categorization are mainly keywords oriented, which can not deal with synonyms

关 键 词： 文本分类 类别核心词 朴素贝叶斯 概念映射 知网

领　　域： [自动化与计算机技术] [自动化与计算机技术]