Presentation at the workshop on ontology quality at EKAW 2010, on using measures of agreement, disagreement, consensus and controversy to support ontology assessment in ontology engineering.

1. Visualizing Consensus with Online Ontologies to Support Quality in Ontology Development Mathieu d’Aquin and Enrico Motta Knowledge Media Institute, The Open University,UK

2. Ontologies are knowledge artifacts, they express opinions and beliefs and contradict each others Assessing (dis)agreement in ontologies is very useful to understand how to combine knowledge from different sources In [1], we defined measures of agreement, disagreement, consensus and controversy between ontologies and ontological statements Here, we show how such measures can be useful in providing an overview of an ontology in development, with respect to the “opinion” of other ontologies on the Web Agreement and Disagreement in Ontologies [1] d’Aquin, M.: Formally measuring agreement and disagreement in ontologies. In: International Conference on Knowledge Capture - K-CAP. (2009)

3. Agreement(st, O)  [0..1] and Disagreement(st, O)  [0..1] where stis a statement <subject, predicate, object> and O is an ontology Based on extracting the part of the ontology that expresses a relation between subject and object (Dis)agreement between ontologies: Global (dis)agreement in a repository Consensus: Controversy: Agreement and Disagreement - Measures

4. What can we do with that? Consensus and controversy in statements about SeaFood in Watson Agreements and disagrements between ontologies containing SeaFood in Watson

5. Vegan subClass Vegetarian SeaFoodsubClassOf Meat SeaFooddisjointWith Meat

6. Agreement, disagreement, consensus and controversy for ontology engineering Making use of these measures to support ontology engineering By providing an overview of the developed ontology introducing the “perspective of other ontologies” I.e., using Watson to measure the global agreement/disagrement and consensus/controversy with online ontologies. And show that in an easy to use visualization.

7. Visualization We show the values of the 4 measures wrt to Watson ontologies using color coding on the edges of a graph representing the statements in the ontology Node colors correspond to the average color of their incoming and outgoing edges Color=agreement/disagreement blue=agreement red=disagreement green=don’t know Brightness=consensus/controversy bright blue=+consensus bright red=-consensus dark=controversy

8. Example: AKT Support Bright blue = high positive consensus Example: Duration subClassOfIntangibleThing Bright red= low negative consensus Example: Duration subClassOfPhysicalQuantity (Cyc, SUMO and ATO_Ontology disagree as they do not declare such a relation, or even say these two classes disjoint)

9. Example: Local Drama Ontology Node color = aggregation of edges colors Example: Organization where there is agreement (Company is a subClass) and disagreement (Archive is a subClass) Green = we don’t know as no other ontology declares anything about two entities Character subClassOf Person

10. The brighter the blue the higher the positive consensus (higher agreement) The brighter the red the lower the negative consensus (higher disagreement) Dark = controversy: no clear cut between disagreement and agreement Example: The statement attached to the class Employee are controversial: some ontologies agree, others disagree (often due to alternative representations of roles) Using consensus to assess an ontology(a new NeOn toolkit plugin AKT Portal

11. NeOn Toolkit Plugin

12. Conclusion and Future Work Visualizing agreement/disagreement and consensus/controversy can be very useful to identify parts of the ontology where a stronger focus is needed in ontology engineering Initial results show that, even in well designed ontologies questions can emerge from such visualizations The tool need a lot of technical work to make it more usable in terms of: Performance (the measures are hard to compute and depend on internet connection) Interaction with the user (e.g., give better explanations for the results)  A complete user evaluation showing how this can guide the “refinement” of an ontology and improve its quality

13. Thank You! m.daquin@open.ac.uk @mdaquin