A presentation describing my final project for an engineering degree at the Hebrew University of Jerusalem - a system for extracting information from web sites into instances of an XML schema, utilizing machine learning, structural analysis of documents and a divide & conquer strategy.

1. WEB INFORMATION EXTRACTION FOR THE DB RESEARCH DOMAIN Michael Genkin (mishagenkin@cs.huji.ac.il) Liat Kakun (liat.kakun@mail.huji.ac.il) School of Engineering and Computer Science Advisor: Dr. Sara Cohen

4. Methods – Structural Analysis #1 Before: After: Transform each input document into a structurally valid, monolithic, document – using industry standard tools such as HTML Tidy and Readability.

6. Methods - Classification Employ multiclass classification (by vector similarity) to map the logical document blocks to the appropriate schema elements.

7. Methods – Pattern Recognition Pattern: .//bibliography/ul/li/* Mine likely candidate blocks for patterns using the PAT Tree algorithm; adjusted for finding a maximum likelihood pattern.

8. Methods – Metadata Extraction Use CRF for extraction of additional metadata where appropriate (e.g. bibliographic lists).

10. Results – Measures

11. Results Precision Recall Pattern Recognition 85% 89.7% Classification Accuracy 82.5%

13. Questions?