作　　者： ; ;

机构地区： 暨南大学信息科学技术学院

出　　处： 《微计算机信息》 2009年第15期123-124,69,共3页

摘　　要： 无论是通用搜索还是垂直搜索,其关键的核心技术之一就是网络爬虫的设计。本文结合HTMLParser信息提取方法,对生活类垂直搜索引擎中网络爬虫进行了详细研究。通过深入分析生活类网站网址的树形结构的构架,开发了收集种子页面URL的模拟搜索器,并基于HTMLParser的信息提取方法,从种子页面中提取出与生活类主题相关的目标URL。经实验测试证明该爬虫的爬准率达93.552%,爬全率达96.720%,表明该网络爬虫是有效的,达到中等规模的垂直搜索企业级应用的要求。 Whether general search engine or vertical search engine, the design of web crawler is the core technology. In this article, a novel system of life-theme web crawler based on HTMLParser information extraction is thoroughly studied. In this system, a simulation searcher is designed for collecting the seed URL by analyzing tree structure of life-theme website, then, based on the discussion of HTMLParser information extraction, the target URL that relate to life-theme is extracted from the seed pages. Empirical studies show that the Precision=93.552% and the Recall=96.720% , proving its effectiveness and achieving requirements for general enterprise-level application of vertical search engine.

关 键 词： 网络爬虫 垂直搜索

领　　域： [自动化与计算机技术] [自动化与计算机技术]