Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie December 2, 2015: NISO/NFAIS Virtual Conference: Semantic Web: What's New and Cool
Ähnlich wie December 2, 2015: NISO/NFAIS Virtual Conference: Semantic Web: What's New and Cool (20)
Mehr von DeVonne Parks, CEM
Mehr von DeVonne Parks, CEM (13)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
December 2, 2015: NISO/NFAIS Virtual Conference: Semantic Web: What's New and Cool
- 1. NISO/NFAIS Virtual Conference: Semantic Web: What's New and Cool, 2 December 2015 Looking Inside the Library Knowledge Vault Jeff Mixter Software Engineer, OCLC Research Bruce Washburn Consulting Software Engineer, OCLC Research
- 2. • Describing the Google Knowledge Vault • Considering how the Knowledge Vault could apply to Library data • Taking a closer look at how MARC record data is transformed into Linked Data • Touring the experimental EntityJS application, for discovery of entities through the Library Knowledge Vault • Summarizing our experimentation to date, and where we’re headed An Overview of Work in Progress
- 3. A series of recent Google Research papers describe the use of probabilistic models and machine learning to assess the truth of statements made by multiple sources. • Li, X., Dong, X. L., Lyons, K., Meng, W., Srivastava, D. (2013). Truth Finding on the Deep Web: Is the Problem Solved? • Dong, X. L., Gabrilovich, E., Heitz, G., Horn, W., Murphy, K., Sun, S., Zhang, W. (2013). From Data Fusion to Knowledge Fusion. • Dong, X. L., Murphy, K., Gabrilovich, E., Heitz, G., Horn, W., Lao, N., ... & Zhang, W. (2014). Knowledge Vault: A Web-scale approach to probabilistic knowledge fusion • Dong, X. L., Gabrilovich, E., Murphy, K. Dang, V., Horn, W., … & Zhang, W. (2015). Knowledge-Based Trust: Estimating the Trustworthiness of Web Sources Estimating Trustworthiness and Finding Truth
- 4. Understanding “RDF Triples” A triple is a statement that relates one thing to another, specifying a Subject, Predicate, and Object. RDF triples use URIs for those three elements. Subject Predicate Object https://viaf.org/viaf/520 10985 https://schema.org/birth Place https://id.worldcat.org/f ast/1204916 Barack Obama Was born in Honolulu, Hawaii
- 5. 1 -- Extractors The 3 Main Components of the Google Knowledge Vault Threshing the Crop, 1480 https://www.flickr.com/photos/marceldouwedekker/7241332380/
- 6. 2 – Graph-based Priors The 3 Main Components of the Google Knowledge Vault Students at Library reference desk at University of Illinois at Chicago Navy Pier Campus. https://www.flickr.com/photos/uicdigital/15578872696/
- 7. 3 – Knowledge Fusion The 3 Main Components of the Google Knowledge Vault Hollerith Census Machine Dials https://www.flickr.com/photos/mwichary/2632673143/
- 9. • OCLC research scientists and software engineers are evaluating a similar model for bibliographic and authority data sources, • in combination with user-contributed content and Linked Data from other providers, • to evaluate a “knowledge vault” for statements about entities and their relationships, including people, groups, places, events, concepts, and works. A “Knowledge Vault” for Libraries?
- 10. Data Sources Extraction Knowledge Vault data flow Extractor Extractor Extractor Data source Data source Data source
- 11. Data Sources Extraction Knowledge Triples Knowledge Vault data flow Extractor Extractor Extractor Graph- based Priors Data source Data source Data source
- 12. Data Sources Extraction Scored Triples Fusion Knowledge Vault Data source Data source Data source Knowledge Vault data flow Extractor Extractor Extractor Fusers Graph- based Priors Knowledge Triples
- 13. A Real World Object
- 14. A creative work, in MARC LEADER 00000nam 2200361 a 4500 001 54881929 008 041130s2004 nyu 000 1 eng 010 2004047063 020 0374153892 (hc. : alk. paper) 040 DLC|cDLC|dYDX 050 00 PS3568.O3125|bG55 2004 082 00 813/.54 |2 22 090 PS3568.O3125|bG55 2004 100 1 Robinson, Marilynne. 245 10 Gilead /|cMarilynne Robinson. 250 1st ed. 260 New York :|bFarrar, Straus and Giroux,|c2004. 300 247 p. ;|c22 cm. 650 0 Conflict of generations |v Fiction. 650 0 Reminiscing in old age |v Fiction. 650 0 Children of clergy |v Fiction. 650 0 Fathers and sons |v Fiction. 650 0 Grandfathers |v Fiction. 650 0 Clergy |v Fiction. 651 0 Kansas |v Fiction.
- 15. LEADER 00000nam 2200361 a 4500 001 54881929 008 041130s2004 nyu 000 1 eng 010 2004047063 020 0374153892 (hc. : alk. paper) 040 DLC|cDLC|dYDX 050 00 PS3568.O3125|bG55 2004 082 00 813/.54 |2 22 090 PS3568.O3125|bG55 2004 100 1 Robinson, Marilynne. 245 10 Gilead /|cMarilynne Robinson. 250 1st ed. 260 New York :|bFarrar, Straus and Giroux,|c2004. 300 247 p. ;|c22 cm. 650 0 Conflict of generations |v Fiction. 650 0 Reminiscing in old age |v Fiction. 650 0 Children of clergy |v Fiction. 650 0 Fathers and sons |v Fiction. 650 0 Grandfathers |v Fiction. 650 0 Clergy |v Fiction. 651 0 Kansas |v Fiction. Strings for Real World Objects
- 16. Identifiers for Real World Objects <http://www.worldcat.org/oclc/54881929> # Gilead : a novel a schema:Book, schema:CreativeWork ; library:placeOfPublication <http://id.loc.gov/vocabulary/countries/nyu> ; schema:about <http://dewey.info/class/813.54/e22/> ; schema:about <http://id.worldcat.org/fast/946347> ; # Grandfathers schema:about <http://id.worldcat.org/fast/921899> ; # Fathers and sons schema:about <http://id.worldcat.org/fast/855328> ; # Children of clergy schema:about <http://id.worldcat.org/fast/864014> ; # Clergy schema:about <http://id.worldcat.org/fast/874815> ; # Conflict of generations schema:about <http://id.worldcat.org/fast/1094438> ; # Reminiscing in old age schema:about <http://id.worldcat.org/fast/1204323> ; # Kansas. schema:author <http://viaf.org/viaf/16729> ; # Marilynne Robinson schema:exampleOfWork <http://worldcat.org/entity/work/id/878639> ; schema:workExample <http://worldcat.org/isbn/9780374153892> ;
- 17. Related Entities schema.org/Place <http://id.loc.gov/vocabulary/countries/nyu> # nyu <http://id.worldcat.org/fast/1204323> # Kansas. schema.org/Intangible <http://dewey.info/class/813.54/e22/> # 813.54$222 <http://id.worldcat.org/fast/1094438> # Reminiscing in old age <http://id.worldcat.org/fast/855328> # Children of clergy <http://id.worldcat.org/fast/864014> # Clergy <http://id.worldcat.org/fast/874815> # Conflict of generations <http://id.worldcat.org/fast/921899> # Fathers and sons <http://id.worldcat.org/fast/946347> # Grandfathers schema.org/Person <http://viaf.org/viaf/16729> # Marilynne Robinson
- 18. Extracting triples from record- oriented data MARC Record Enhanced WorldCat MARC Record MARC Records • FRBR Clustering • String matching with controlled vocabularies • Addition of standard identifiers
- 19. MARC Record Enhanced WorldCat MARC Record Persons Organizations Places Concepts Events Works MARC Records RDF Entities • FRBR Clustering • String matching with controlled vocabularies • Addition of standard identifiers Extracting triples from record- oriented data
- 20. MARC Record Enhanced WorldCat MARC Record Persons Organizations Places Concepts Events Works MARC Records RDF Entities Triples • FRBR Clustering • String matching with controlled vocabularies • Addition of standard identifiers Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Subject Predicate Object Extracting triples from record- oriented data
- 21. MARC Records and linked data challenges • A single record may contain many subjects: people, organizations, places, topics, and events • The relationships between these subjects isn’t always clear • These entities are not always “notable” … they may lack identifiers in library authority systems (if not affiliated with published works) and lack identifiers elsewhere (if not notable enough to warrant a Wikipedia article)
- 22. Creating a Library Knowledge Vault • Triples in a library knowledge vault provide opportunities for applications supporting discovery, editing, visualization, and more • OCLC Research is investigating what it’s like to assemble and work with this kind of data in an experimental discovery system we call “EntityJS”
- 23. The EntityJS Research Project Assessing the usability of Linked Data, testing entity refinement and editing, and contributing to the Knowledge Vault.
- 24. Testing with a subset of Knowledge WorldCat MARC records describing archival materials Enhanced MARC records
- 25. Knowledge Triples Scored Triples Testing with a subset of Knowledge WorldCat MARC records describing archival materials Enhanced MARC records Extractor Extraction
- 26. Knowledge Triples Scored Triples Testing with a subset of Knowledge WorldCat MARC records describing archival materials Vault Services EntityJS Enhanced MARC records Extractor Extraction
- 27. Knowledge Triples Scored Triples WorldCat Testing with a subset of Knowledge WorldCat MARC records describing archival materials Vault Services EntityJS Wikidata DBPedia VIAF FAST Enhanced MARC records Extractor
- 28. Knowledge Triples Scored Triples WorldCat Testing with a subset of Knowledge WorldCat MARC records describing archival materials Vault Services EntityJS Application Triples Wikidata DBPedia VIAF FAST Enhanced MARC records Extractor Extraction
- 29. Knowledge Triples Scored Triples Knowledge Vault WorldCat Testing with a subset of Knowledge WorldCat MARC records describing archival materials Vault Services EntityJS Application Triples Wikidata DBPedia VIAF FAST Fusers Enhanced MARC records Extractor Extractor Extraction
- 34. User-contributed “same as” relationships
- 35. User-contributed “same as” relationships INSERT DATA { GRAPH <http://id.worldcat.org/fast/1405559> <http://schema.org/sameAs> <http://www.wikidata.org/data/Q502093>. }
- 36. User-contributed “same as” relationships
- 37. Extractors Collective Knowledge Triples Scored Triples Fusion Knowledge Vault WorldCat An end-to-end test of the Knowledge Vault Vault Services EntityJS Application Triples Wikidata DBPedia VIAF FAST Fusers Enhanced MARC records Extractor Extractor
- 38. Continued Experimentation • Build a way to assign confidence levels to data contributed by EntityJS • Use confidence levels as input to a Fusion process to created Scored Triples • Extend the EntityJS application to incorporate additional Linked Data resources and support further entity relationship refining and editing
- 39. More to think about … Machine Learning: the High-Interest Credit Card of Technical Debt D. Sculley et al, Google Inc. http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43146.pdf Entanglement: Machine learning models are machines for creating entanglement and making the isolation of improvements effectively impossible. Hidden Feedback loops: Gradual changes not visible in quick experiments make analyzing the effect of proposed changes extremely difficult. Undeclared Consumers: They may be difficult to detect unless the system is designed to guard against them, and can introduce additional hidden feedback loops.
- 40. SM Contact us Jeff Mixter Software Engineer, OCLC Research mixterj@oclc.org Looking inside the Library Knowledge Vault Bruce Washburn Consulting Software Engineer, OCLC Research bruce_washburn@oclc.org
Hinweis der Redaktion
- This presentation describes work in progress at OCLC, in which we are investigating how the “Knowledge Vault” described by Google Researchers might apply to bibliographic and authority data created and managed by libraries. Jeff Mixter and Bruce Washburn are software engineers in OCLC’s Research Division, working with others across the organization on this investigation. OCLC is a global library cooperative that provides shared technology services, original research and community programs for its membership and the library community at large. We are librarians, technologists, researchers, pioneers, leaders and learners. With thousands of library members in more than 100 countries, we come together as OCLC to make information more accessible and more useful.
- Today we’re going to talk about some work in progress in OCLC’s Research division. We’ll begin by covering some recent research work at Google that describes the processes they are applying to build a highly-scalable Knowledge Vault, in which competing statements about things described on the web are assessed to determine the level of confidence in the source and the level of certainty about the statement’s truthfulness. We’ll then talk about how in OCLC Research we’re thinking about that research work, and applying it in experimental ways with Library data. As part of our work we’ve created an experimental application that we’re calling EntityJS, which can explore data through the Library Knowledge vault. We think it’s an interesting entity-oriented Linked Data discovery app, and would like to get your input on that. And, as this is a work in progress and still in a formative stage, we’ll summarize where we are at and where we’re headed next.
- Google researchers have recently published a series of papers describing how they use probabilistic models and machine learning to assess the truth in the statements made about things and their relationships, mined from the web. Google researchers refer to this as the Knowledge Vault. OCLC Research Scientists and engineers have been taking a close look at this work, and thinking about how it may apply to how we aggregate, extract, and fold together bibliographic and authority data from a wide range of sources, as we pursue the work of building linked data representations for the things and the relationships those sources describe.
- Before getting too far ahead of ourselves, let’s contend with some necessary jargon and define an important concept … RDF Triples. We’ll be referring to “triples” a lot in this presentation. A “Triple” is simply a statement that relates one thing to another. It has 3 parts, a subject, predicate, and object. When working with large and varied data sources we need a common mechanism for comparing and sifting the aggregated data, and a way to relate one thing to another. The RDF Triple gives us a common way to do that. RDF stands for Resource Description Framework, a way of modeling data, and an “RDF Triple” is a triple that has identifiers for the subject, predicate, and object. In this example we see a VIAF identifier for Barack Obama, a schema.org identifier for place of birth, and an OCLC FAST identifier for Honolulu Hawaii, expressing a factual claim about the President’s place of birth. The triple can be read like a very simple English sentence: “Barack Obama was born in Honolulu, Hawaii.”. These identifiers have an important quality of being “dereferenceable” … that is, they can be used with an internet protocol, like HTTP on the web, to look up more information about the thing they represent, and they are language-independent.
- “Extractor systems extract triples from a huge number of Web sources. Each extractor assigns a confidence score to an extracted triple, representing uncertainty about the identity of the relation and its corresponding arguments.”
- “Graph-based Prior systems learn the prior probability of each possible triple, based on triples stored in an existing Knowledge Base.”
- “The Fusion system computes the probability of a triple being true, based on agreement between different extractors and priors.”
- Recalling the earlier example of a triple indicating that Barack Obama was born in Honolulu … Based on the many data sources Google consumes, they may find information about Barack Obama that identifies his birthplace as Kenya … perhaps the extraction is actually referring to his father, or perhaps the site is making an factually incorrect claim. Prior information about this fact in existing knowledge bases can be used to determine the level of uncertainty about a new or conflicting claim. And if the knowledge base did not indicate a birth place for the person, it may have other information about the person (in this case, his role as President of the United States) to help determine the level of uncertainty in a new claim.
- While recognizing that the web of data that Google is attempting to assess encompasses a much larger set of sources, and represents a higher frequency of competing “claims” of truth about entities and their relationships, we see real potential in applying their approach to the data sources we’ve traditionally worked with.
- The first step in the process is extract “triples” from record-oriented data. Triples are simple statements that describe a relationship between a subject and an object. For example, the creative work Persuasion was created by Jane Austen. Expressing those simple statements in RDF, the standard model for information interchange on the web, provides a way to aggregate and process triples provided by a wide range of sources.
- The “Extractors” produce “Knowledge Triples” which can associate the statement with who produced the statement, when it was produced, and other properties that can be used later on to compare related claims. While the data sources that serve as input to the extractors can take a wide variety of forms … scraping data from HTML pages, XML documents, JSON documents, etc. … and may reflect a wide range of schemas and ontologies … Dublin Core, MARC, EAD, Wikidata, etc. … the knowledge triples are normalized to adhere to a consistent vocabulary. In our work, that vocabulary is Schema.org. This agreement is not unlike our everyday world, where communication depends upon the agreement to use a shared vocabulary.
- The “Fusion” processes apply mathematics to the collective of Knowledge Triples, to produce Scored Triples in the Knowledge Vault. In our current state of development, we’re still just experimenting with the Fusion step … more about that in a bit.
- Here’s a brief side trip to consider the book as a Real World Object, and how it and its relationships to other real world objects can be expressed as Linked Data.
- If you’re familiar with MARC cataloging, you’ll be familiar with this view. The MARC Record might be thought of as a mashup of lists. There are lists of attributes associated with the book in the real world … it’s title, pages, and size. There are lists of attributes about it as one manifestation of a conceptual work … call numbers, edition statements, who published it, and when and where. And there are lists of other entities that played some role in the creation of the work … authors, editors, translators … and conceptual entities that can be used to describe what kind of a work it is (fiction vs. non-fiction) and what it’s about. Combined in this way, the record provides a useful wrapper for multiple purposes … originally, for printing catalog cards, but more recently for supporting cataloging, resource sharing, online discovery, and more.
- Looking at the record we see within its structure various strings of text that described real world objects
- If those strings can be matched to controlled vocabularies and knowledge bases that represent the string with a persistent URI, the associated entities and their relationships begin to emerge as Linked Data
- With instances of entities belong to classes of various types
- For our experimental work, we begin with data that is record-oriented. MARC bibliographic and authority data. As part of the normal operations of OCLC these records are clustered following the FRBR principles, and a significant amount of additional work goes into matching strings representing people, groups, places, events and concepts to corresponding authoritative headings in controlled vocabularies, with identifiers from those vocabularies (LCSH, VIAF, FAST, and others) assigned where possible.
- Our next step is to reconstitute the record-oriented data to represent entities … OCLC provides Work Entities as Linked Data now, and in our experimentation we’re testing these representations for a total of 6 types of entities. The entity data is modeled using RDF … The Resource Description Framework … and for users of the data it can then be represented in a variety of Linked Data serializations, including JSON-LD, N-Triples, TTL, or RDF XML.
- And from those Entities we derive Triples representing the relationships between things that are described in the entity data. This processing of records into entities and then into triples increases the granularity of the information objects … 4 million records are converted into about 40 million separate entity descriptions, which yield about 140 million triples. This process includes an action that is similar to the “Graph-based Priors” application of existing Knowledge Bases to validate or extend data Google harvests from the web. The source MARC records include undifferentiated headings for people, places, etc., lacking standard identifiers, and sometimes taking alternative string representations. Using vocabularies like VIAF and FAST as Knowledge Bases, and taking into account other information in the record and related records in a FRBR cluster, these varying strings can be related to the same entity and inherit the richness of that entity description.
- This conversion process does have some challenges. Specifically, a single bibliographic record might include many subjects, please, organizations, places, topics and events. Consequently the conversion of a single record can result in many different Entities that need to be accounted for, created and subsequently managed. Secondly, due to some of the limitations of the MARC standards, the relationships between entities within a record or entities to the corresponding Creative Work entity are not always clear. For example a MARC record might have an 100 field which denotes an creator but unless specific indicators and subfields are used, we are not able to determine if the relationship is ‘author’, ‘painter’, ‘photographer’ etc. Consequently in this instance, we would have to assume a generic ‘creator’ relationship. It is also difficult to know the relationship between individual subjects in a MARC record. If a MARC records has a 600 subject field with ‘Abraham Lincoln’ and a 650 subject field with ‘Presidents’ we can not tell what the relationship is, or even if there is one, between the two subjects. A human could clearly make the relationship but conversion algorithms can not. Finally, there are many times when entities are not “notable enough” to warrant existence in an Authority file. Since we try to derive URIs for entities from existing Authority files this would cause issues with the conversion process. We have a work-around, which is to coin a new URI for the entity but this then creates a new workflow and management pipeline (i.e. Entity creation, management and maintenance) that needs to be implemented.
- The goal of our Knowledge Vault work is to create a store of data that can be used in discovery and allow for editing, visualization and more. To help test the idea start to finish, OCLC Research has been working on an experimental Entity discovery system called “Entity JS”. The JS part stands for Java Script. As part of the project, we were interested in conforming to and using modern/emerging pieces of infrastructure. Specifically we wanted to highlight what client side applications were capable of. As a result we built Entity JS using the Angular.js framework.
- Project Goals Prototype an application that runs in a browser and uses RDF data sources from OCLC and elsewhere Write as little original code as possible; use existing code libraries Skip the magic tricks that may hide Linked Data issues Search across entities Show relationships of one entity to others Examine questions around user-contributed improvements to entity relationships Evaluate how an application like EntityJS could both consume data from and contribute data to the Vault Why the name “EntityJS”? We’re thinking about entities, so included that in the name. The “JS” stands for “JavaScript”, the programming language we’re using for our experimental development work. Some common software frameworks we’re using have applied, the “JS” suffix as well … the AngularJS client application software, NodeJS and ExpressJS for server applications … so we’re following that pattern. Keep in mind that this is just an experimental application, for now, and we just needed a name when referring to it. We’ll probably call it something else, if it grows up to be something more than an experiment.
- To examine the process of transforming bibliographic records to triples and creating a knowledge vault, we’ve extracted a subset of WorldCat records … those that describe the primary source materials that make up the OCLC Research ArchiveGrid discovery system. ArchiveGrid includes about 4 million records from WorldCat describing primary source materials … unpublished letters, diaries, scrapbooks, films, recordings, photographs, etc. … found in the collections of archival institutions and special collections libraries.
- Though WorldCat is our only library data source in the testing phase, we’re working with an “enhanced” version of the data provided by WorldCat catalogers, where FAST and VIAF identifiers have been automatically added (where possible) to supplement the strings representing people, groups, places, concepts, and events. The term “vault” suggests that the triples are valuable (which is true) but also that they are hidden or inaccessible (which is not the intent). We imagine that vault services will be built that ingest and process these triples in various ways. For example, they might be processed into indexes to support fast search and retrieval of specific entities and their relationships
- We’ve developed a few services to provide access to the scored triples in the vault. We use our Hadoop cluster and MapReduce scripts to convert the triples into data that can be indexed, and use ElasticSearch for supporting that index, so that we can quickly search across the data. We have also loaded the scored triples into a Triplestore and run server scripts for sending SPARQL queries to that Triplestore, to retrieve unindexed data. EntityJS interacts with these Vault services to search across entities, and to view individual entities and their relationships. When viewing a single entity the application connects directly to its RDF source. For example, to initiate the view of a person, the EntityJS client application retrieves RDF XML data from viaf.org.
- The EntityJS application makes real-time use of a range of other Linked Data resources, including WorldCat Works, VIAF, FAST, DBPedia, Wikidata, GeoNames and others, to enrich the display.
- EntityJS can also be used to create new relationships between entities found by searching the knowledge vault, and store temporary versions of that data for immediate use by the EntityJS application.
- EntityJS can also be used to produce new data to flow back into the Knowledge Vault process … this is our opportunity to experiment further with the Knowledge Fusion process, as we can use different confidence scores assigned to the knowledge triples produced by EntityJS than those derived from WorldCat’s ArchiveGrid subset, and test how that affects use of the data in the Knowledge Vault.
- Based on triples in our knowledge vault we built an ElasticSearch engine that helps us quickly search across entities.
- When an entity is selected, we base the initial view of it on its RDF data source (in this case, a person entity represented by a VIAF record). We supplement that with information represented by knowledge vault triples, displaying related works and the role this person has associated with that work (creator, contributor, or subject). We also find other related entities and show those that are identified most frequently in the set of ArchiveGrid source records … a kind of “PageRank” to show the most closely related entities first.
- Relations to other representations of the entity, as in WikiData, help us supplement the list of alternate names for the person …
- And can identify family relationships to other people, not all of whom would have been otherwise represented in the initial set of ArchiveGrid MARC records. The family relationships from WikiData provide another network layer on top of the relationships in the MARC data.
- Not all of our bibliographic or authority records contain connections to related forms. Here is an example of a FAST subject that could benefit from a connection to the corresponding WikiData page. EntityJS offers a way for users to select from a set of potentially related WikiData/Wikipedia articles for the subject.
- Selecting a matching subject from the list sends an instruction to the EntityJS knowledge vault services to remember that as a “same as” relationship …
- … and the data and related images from Wikidata and Wikipedia can immediately enrich the view in the EntityJS application, while going back into the mix to compete with other factual claims in the process that creates the knowledge vault.
- Here’s where the EntityJS application begins to help us further explore the capacity of the knowledge vault … introducing a new set of statements, knowledge triples, that can contend with similar claims from other library data sources and be “fused” through the authority of their sources and their past history in knowledge bases to become a scored triple in the knowledge vault. A full loop through the extraction and fusing process.
- The next steps in our experiment will be to build out and test the infrastructure to assign confidence scores to data contributed by Entity JS. We have a test implementation working where authenticated users have a default score of 7 and un-authenticated users have a default score of 5. We would like to expand this to include user profiles that can be used to create dynamic scores. For example a NACO certified cataloger or cataloger from a NACO certified organization might have higher scores assigned to changes they make about names of Entities Up to now we have not really tested the fusion process. As we begin to aggregate more data, either from human input or from new bulk data streams, we will need to figure out how to extract and score the data and then fuse it into the ‘Vault’ in a logical and efficient way. As the work on the Entity JS application continues, we would like to add in more datasets to help test the agnostic-ness of the technology. We would also like to add support for additional entity editing such as literal editing and adding addition triples.
- As we pursue this experimentation, we’re mindful that we’re going to encounter new, for us, concerns about how machine learning can effect, undermine, skew or distort what we’re aiming towards: truthfulness. This Google Research paper identifies those concerns in a helpful and readable way.