In this paper, we introduce the concept of a QA-Pagelet to refer to the content region in a dynamic page that contains query matches. We present THOR, a scalable and efficient mining system for discovering and extracting QA-Pagelets from the Deep Web. A unique feature of THOR is its two-phase extraction framework. In the first phase, pages from a deep web site are grouped into distinct clusters of structurally-similar pages. In the second phase, pages from each page cluster are examined through a subtree filtering algorithm that exploits the structural and content similarity at subtree level to identify the QA-Pagelets.
Citation:
James Caverlee, Ling Liu, David Buttler, "Probe, Cluster, and Discover: Focused Extraction of QA-Pagelets from the Deep Web," icde, pp.103, 20th International Conference on Data Engineering (ICDE'04), 2004