Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie PreMapper: Improving Entity Extraction Accuracy in the Digital Humanities
Ähnlich wie PreMapper: Improving Entity Extraction Accuracy in the Digital Humanities (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
PreMapper: Improving Entity Extraction Accuracy in the Digital Humanities
- 1. IBM Haifa Research Lab PreMapper: Improving Entity Extraction Accuracy in the Digital Humanities Cormac Hampson (cormac.hampson@scss.tdc.ie) Ella Rabinovich (ellak@il.ibm.com), Sara Porat (porat@il.ibm.com) Maya Koleva (maya.koleva@commetric.com), Ivan Uzunov (ivan.uzunov@commetric.com) Owen Conlan (owen.conlan@scss.tcd.ie) © 2014 IBM Corporation
- 2. IBM Haifa Research Lab What is CULTURA • Digital humanities portal supporting the exploration of cultural heritage collections by a range of different users • • • Professional researchers and historians Students with little or no experience of a particular archive There are three digitised collections in the portal • 1641 Depositions (http://cultura-project.eu/1641/) • Bureau of Military History - 1916 Rising (http://cultura-project.eu/1916) • IPSA Collection (http://cultura-project.eu/ipsa) 2 © 2014 IBM Corporation
- 3. IBM Haifa Research Lab Smart Content Analysis with Entity-Relationship Extraction • A powerful technique for injecting semantics into unstructured text • • • Employing Natural Language Processing (NLP) Involving training a dataset and/or using prior knowledge (e.g., dictionaries) so that specific entities can be identified within the text Each collection introduces its unique entity-relationship model • Entities, e.g., Person, Location, Event • Entity attributes, e.g., Person.occupation, Deposition.mentioned_date • Relation between entities, e.g., Person at Location 3 © 2014 IBM Corporation
- 4. IBM Haifa Research Lab Entity Extraction – Example <title> first-name last-name sir Robert Andrew 4 © 2014 IBM Corporation
- 5. IBM Haifa Research Lab Manual Updates of Extracted Entities - Motivation • The automatic task of entity extraction cannot provide full accuracy • The 1641 Depositions collection introduces additional difficulty due to the noisy text, inconsistent grammar and spelling • Extraction errors can damage a curator’s trust in the automatic processing, as well as an end user’s overall confidence in the system • Approaches to improve the accuracy of entity extraction are of major benefit of the CULTURA environment 5 © 2014 IBM Corporation
- 6. IBM Haifa Research Lab Entities Visualisation and Modification with PreMapper • PreMapper is a web-based visualization and analysis tool that is integrated into the CULTURA environment • Provides visualisation and editing of entities, maps, flows and relationships between individuals and groups • Entities (people, organizations) are represented by nodes, links present relationships between these nodes 6 © 2014 IBM Corporation
- 7. IBM Haifa Research Lab Manual Changes of Extracted Entities PreMapper enables curators of the collection to make manual changes to the extracted entities using a GUI • Add/delete/update entity • Merge two entities into a single entity (entities disambiguation) • Add/delete relationship between entities The entity “sir phelim” can be merged with the entity “phelim neil” if an expert deems that these entities refer to the same person 7 © 2014 IBM Corporation
- 8. IBM Haifa Research Lab General Flow 8 © 2014 IBM Corporation
- 9. IBM Haifa Research Lab Entity Disambiguation via PreMapper • The task of determining the identity of entities mentioned in the text • • e.g., based on entity’s key attributes Entity disambiguation in historical content is one of the main challenges of CULTURA professional users • • Are “Rob. Meredith” and “Robert Meredith” the same person? • • Are “sir Phelim” and “Phelim o neil” the same person? Entities scope matter (disambiguation of entities found in the same deposition vs. entities found in different depositions) Non-functional challenges • Authorization – who is allowed to make changes? • Personalization – what is the scope of a specific change (specific researcher, group of researchers, the entire professional community)? • Verification – who verifies the changes? 9 © 2014 IBM Corporation
- 10. IBM Haifa Research Lab Summary and Future Work • Entity-relationship extraction is a powerful technique for extracting structured information from unstructured documents • PreMapper is a visualization tool that allows domain experts to improve the accuracy of the entity-relationship data • Domain experts feedback is important in refining the user interface with the CULTURA environment • • It becomes vital when entity extraction is error-prone, as with the 1641 Depositions collection that contains a lot of noise and misspellings Future work includes further exploration and design of the fully integrated end-to-end solution http://staging1.commetric.com:8080/cultura/?q=1641&ids=836062r034&nodeTypeId= 7&layout=circle#svg-graph-editor-switch 10 © 2014 IBM Corporation
- 11. IBM Haifa Research Lab 11 © 2014 IBM Corporation
Hinweis der Redaktion
- &lt;number&gt; 1641 Depositions – witness testimonies, mainly by Protestants, but also by Catholics concerning their experience during the 1641 Irish rebellion. These testimonies document different incidents like robbery, military actions, imprisonment and murder. 1916 Rising – documents events that happened during a rising that occurred at Enniscorthy (Ireland) in 1916. IPSA collection – a digital archive of illuminated medieval manuscripts, with high quality drawings and metadata (mainly herbs and astrology).
- &lt;number&gt; In our work entity extraction involved definition of dictionaries for each entity type we wanted to extract (e.g. FirstName, LastName, Location, Date), and a set of parsing rules that use these dictionaries to annotate piece of text that presumably fits an entity, or entity’s attribute and relationship between entities.
- The parsing rule for identification of this Person includes person’s title (e.g., sir, Mrs), then FirstName followed by LastName, or LastName followed by FirstName. We have a few tens of these rules for a comprehensive and accurate (to some extent) annotation of text.
- The main part of the paper of therefore this presentation focuses on that task of manual updates of extracted entities. The task of entity extraction can not produce 100% accuracy even under perfect conditions (such as clean and well structured text); it becomes much more difficult when considering 1641 Depositions collection, with noisy text, huge number of misspellings and inconsistent grammar. As an example, the same person’s name can be written differently in the same deposition; yet we strive to disambiguate these names and conclude that they refer to the same person. Another example are entities attributes, e.g., Person’s occupation, religion or origin identification becomes challenging due to inconsistent structure of sentences. In CULTURA we adopted an approach for manual modifications of extracted information using the PreMapper Tool that I present next.
- &lt;number&gt;
- &lt;number&gt; Note that Entity-Relationship data is used for several (currently decoupled) purposes: (1) entities visualization with the PreMapper tool and (2) Entity-oriented search that is not the subject of this presentation, but in few words is used for retrieval and exploration of extracted entities. Changes in extracted entities should be reflected in the EoS component, that uses the open source Lucene search engine for retrieval. We will focus on the architecture of distributing the modifications made by users in one of the next slides.