QAestro relies on a controlled vocabulary and the Local-as-View (LAV) approach to model QA tasks and components, respectively. We have formalized 51 existing QA components implemented in 20 QA systems using QAestro. Our empirical results suggest that QAestro enumerates the combinations of QA components that effectively implement QA developer requirements. Published in Proceedings of DEXA 2017, Lyon, France
7. Problem Statement
● Define a framework able to
○ semantically describe QA components and QA developer
requirements; and
○ produce QA component compositions based on these
semantic descriptions.
7
8. Our Contributions
● The QAestro framework that generates QA component
compositions based on developer requirements.
● A vocabulary for expressing QA tasks and developer
requirements.
● The formalization of existing 51 QA components from 20 QA
systems.
● A mapping of the QA component composition problem into
Query Rewriting Problem (QRP).
● An empirical evaluation of QAestro framework.
8
9. Agenda
1. Approach
a. Controlled Vocabulary for Abstract QA Tasks
b. Semantic Descriptions of QA Component
c. QA Developer Requirement
d. The Problem of QA Component Composition
2. Empirical Study
3. Conclusions
9
11. Controlled Vocabulary for Abstract QA Tasks
● QAV= (𝛿,A)
○ 𝛿: signature of logical language.
○ A: set of axioms describing the relationships among vocabulary
concepts.
11
12. Controlled Vocabulary for Abstract QA Tasks
● QAV= (𝛿,A)
○ 𝛿: signature of logical language.
○ A: set of axioms describing the relationships among vocabulary
concepts.
● Example: Disambiguation(x, y, z, t) → QuestionAnalysis(x, y)
○ where x=entity; y=question; z=disambiguated entities, t=template ;
12
13. Semantic Description of QA Component
● Follow Local as View (LAV) approach to define QA components.
○ Allows easy scaling up to large number of QA components.
● Example:
○ Describe Stanford NER that performs named entity recognition
task.
13
14. Semantic Description of QA Component
● Example:
○ StanfordNER($y,x) :– Recognition(y,x), Question(y), Entity(x)
■ Recognition is the QA task of Stanford NER component.
■ StanfordNER accepts question as input and gives recognised
entities as output.
■ Question, Entity, variables x and y are part of controlled
vocabulary.
14
15. QA Developer Requirement
● Example: Give me all the components that implements entity
recognition and entity disambiguation task.
● Semantic Description of QA Developer Requirement:
○ QADevReq($y, x) :– Recognition(y,x), Disambiguation(x,y,z,t)
15
16. The Problem of QA Component Composition
● How to compose valid compositions using LAV mappings of QA
component.
LAV Mapping Examples:
○ DBpediaNER($y, x) :– Recognition(y, x), Question(y), Entity(x)
○ Alchemy($y, z) :– Disambiguation(x, y, z, t), Question(y), DisEntity(z)
○ Qakisatype($y, a) :– Answertype(y, a, o), Question(y), Atype(a)
○ StanfordNER($y,x) :– Recognition(y,x), Question(y), Entity(x)
○ Agdistis($x, $y, z) :– Disambiguation(x, y, z, t), Entity(x), Question(y),
disEntity(z)
16
20. Experiment Configuration
● QAestro Implementation
○ Implemented in python 2.7 on top of MCDSAT.
○ Source Code: https://github.com/WDAqua/QAestro
○ 51 QA component formalisation for 20 QA systems
● QAestro Experiments
○ Executed on a laptop (with Fedora Linux 25) Intel i7-4550U,
4x1.50GHz and 8GB RAM.
20
25. Conclusions
● Introduce QAestro- a framework that:
○ Semantically describe QA Components and Developer
Requirements.
○ Follows Local as View approach.
○ Compose valid compositions of QA components.
○ Can successfully deal with the growing number of QA.
○ Demonstrates efficient processing time.
25
26. Find us!
● For more information please visit: http://wdaqua.eu/QAestro/
● Demo can be viewed at:
○ https://www.youtube.com/watch?v=9lhamebx7JM&feature
=youtu.be
● Write me on : kuldeep.singh@iais.fraunhofer.de
26
Hinweis der Redaktion
In this slide, I will talk about the problem of QA component orchestrations in a framework.
A signature is a set of predicate
and constant symbols, from which logical formulas can be constructed, whereas
the axioms A describe the vocabulary by illustrating the relationships between
concepts.
A signature is a set of predicate
and constant symbols, from which logical formulas can be constructed, whereas
the axioms A describe the vocabulary by illustrating the relationships between
concepts.
disambiguation is a predicate of arity four in ;
disambig(x; y; z; t) denotes that the QA task .
disambiguation relates an entity x,
a question y, a disambiguated entity z, and a template t. Furthermore, the binary
predicate questionAnalysis(x; y) models the question analysis task and relates
an entity x to a question y.
QA components are defined formally w.r.t their functionality, input and output dependencies
Each component is defined as conjunctive rule considering
All the variables in the head of a rule are also variables in the predicates in the body of the rule.
QA components are defined formally w.r.t their functionality, input and output dependencies
Each component is defined as conjunctive rule considering
All the variables in the head of a rule are also variables in the predicates in the body of the rule.
Talk about Stanford NER has its input y i.e. question
Talk about Stanford NER has its input y i.e. question
Talk about Stanford NER has its input y i.e. question
that in almost half of the QA systems, components that implement Tokenization and Query Generation are included, while some less popular QA tasks like Answer Type Identification and Syntactic Parser are part of only two QA systems.
Talk about Stanford NER has its input y i.e. question
Talk about Stanford NER has its input y i.e. question