STAVIES: A System for Information Extraction from unknown Web Data Sources through Automatic Web Wrapper Generation Using Clustering Techniques
ABSTRACT
In this paper a fully automated wrapper for information extraction from Web pages is presented. The motivation behind such systems lies in the emerging need for going beyond the concept of "human browsing.” The World Wide Web is today the main "all kind of information” repository and has been so far very successful in disseminating information to humans. By automating the process of information retrieval, further utilization by targeted applications is enabled. This novel presents a fully automated scheme for creating generic wrappers for structures web data sources. The key idea in this novel system is to exploit the format of the Web pages to discover the underlying structure in order to finally infer and extract pieces of information from the Web page. This system first identifies the section of the Web page that contains the information to be extracted and then extracts it by using clustering techniques and other tools of statistical origin. STAVIES can operate without human intervention and does not require any training. The main innovation and contribution of the proposed system consists of introducing a signal-wise treatment of the tag structural hierarchy and using hierarchical clustering techniques to segment the Web pages. The importance of such a treatment is significant since it permits abstracting away from the raw tag-manipulating approach. The system performance is evaluated in real-life scenarios by comparing it with two state-of-the-art existing systems, OMINI and MDR.
