PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces

Enhancing Software Quality and Quality of Experience
through User Interfaces
Pedro Luis Mateo Navarro
Advisors:
Dr. Diego Sevilla Ruiz,
Dr. Gregorio Mart´ınez P´erez
Murcia, July 2014
Pedro Luis Mateo Navarro () PhD Thesis Defense Murcia, July 2014 1

Agenda
Enhancing Software Quality and Quality of Experience
through User Interfaces
I. Motivation, Goals, and Context
II. Solutions Proposed in this PhD Thesis
III. Conclusions and Further Work
IV. Contributions Summary

I
Motivation, Goals, and Context

Table of contents
1 Motivation
2 Goals of this PhD Thesis
3 Context of this PhD Thesis

Quality of Software and Interaction
Quality of Software is a broad concept...
encompasses many disciplines: from technical to psychological
can be achieved in many ways:
checking functional requirements of software
evaluating users perceived quality
etc.
This thesis focuses on User Interfaces...
...a ﬁeld in which there is too much work to do

User interfaces
The link between the human and the machine...
users interact with the interface → they can see/touch/feel it
system provide responses through the interface
mean of communication
User interfaces are essential;
guaranteeing their quality is critical

Software Testing
Processes to guarantee software and interfaces quality...
a system or application works as expected?
a system or application works within its context?
guarantee that the requirements are met
Testing is essential before a software is released

Human-Computer Interaction (HCI) Assessment
HCI assessment ⇔ discipline within Software Testing
Study of a conversational process involving:
user → has a task to accomplish; provides input
system → processes data to elaborate a response
context → surrounds them; aﬀects behavior and response
Interaction has changed in the past years

What is Good-Quality Software?
Quality software ⇔ requirements are met
The software fulﬁlls:
functional requirements → the behavior is the expected
non-functional requirements → structural quality
Quality can also be measured in terms of
Usability and Interaction Experience

Software Usability & Quality of Experience (QoE)
Usability: How easy it is to use a software +
how easy it is to learn how to use that software
assure software eﬀectiveness (is it useful?) and eﬃciency
assure user satisfaction
QoE: Subjective measure of users experiences with a software
based on user perception and user feelings
encompasses user behaviors + attitudes + emotions
dynamic → changing context/circumstances

Achieving Good-Quality Software?
Diﬀerent ways to achieve software quality
We try to narrow the problem...
focusing on user interfaces
analysis of user-system interaction in real-time
How can we assess interaction?
Since interaction = user inputs + system outputs then...
...we posed 2 diﬀerent approaches

Achieving Software Quality through User Interfaces
Approach 1: Decompose interaction into its components
main components ⇒ input + output
components quality is enhanced separately
analysis/testing processes → more focused, more eﬀective
Approach 2: Assess interaction process as a whole
whole dialog → keeps totality of the process
conversational nature → cause-eﬀect relationships
analysis of external factors ⇒ context

Table of contents
1 Motivation

Goal of this PhD Thesis
Improve methods for software testing and interaction analysis
design better tools → +eﬀective, +robust
design easy-to-integrate tools → +adaptable, +lightweight
ease the process of achieving software quality
Too abstract/ambitious ⇒
we deﬁned 4 main Goals (2 Blocks)

Research Goals in Block 1
Block 1: assure quality of interaction components separately
Goal 1.1: Ease the validation of system response
automation of GUI testing processes → simulate expert actions
ease the integration into diverse scenarios
Goal 1.2: Allow veriﬁcation of user inputs at runtime
protect GUI data using rules/constraints → prevent malfunction
dynamic and interactive veriﬁcation process

Research Goals in Block 2
Block 2: assure quality of user-system interaction
Goal 2.1: Describe multimodal interaction for its assessment
generic and dynamic description of interaction
allow instrumentation, analysis, and comparison
Goal 2.2: Allow analysis of user experiences in mobile scenarios
combine interaction/context data with users subjective data
generic description of mobile context and users perceived quality

Table of contents
1 Motivation

Context of this research
C´atedra SAES-UMU:
research initiative created by SAES and the University of Murcia
improve development processes
promote the use and development of open-source tools

SAES
SAES (Cartagena, Spain) is the only Spanish company specialized in
submarine electronics and acoustics
Besides funding our work, SAES provides us with:
real problems to be solved
realistic quality requirements to be achieved
real scenarios in which integrate and use the developed methods

Research Internship at TLabs
Telekom Innovation Laboratories = Deutsche Telekom + TU Berlin
(Berlin, Germany)
Central research institute of Deutsche Telekom:
more user-centered approaches
real problems in interaction with current technology

II
Solutions Proposed in this PhD
Thesis

Table of contents
4 Block I
Validating System Output
Verifying Users Input
5 Block II
Evaluating Multimodal Interaction
Evaluating Mobile QoE

Table of contents
4 Block I
5 Block II

Goal 1.1 & Problems Validating System Response
→ Goal 1.1: Ease the validation of system response
Problems with GUI output validation:
expensive/cumbersome processes → automation
high dependency on experts → encapsulation + simulation
changes during GUI design/development → robust testing
diﬃcult integration of testing tools
diverse testing environments → adaptable architecture
code-intrusiveness → non-invasive tools

Solution: Open HMI Tester (OHT)
OHT: a framework to develop GUI-testing tools
→ Capture/replay approach
GUI Testing ⇒ uses test cases to check requirements
1 Capture → create test cases with interaction
experts interact with parts to be tested
GUI introspection → test cases created automatically
2 Replay → execute test cases to validate app response
functional & regression testing
expert simulation: GUI events → +tolerance to changes

HMI TESTER
PRELOAD MODULE
TESTED APPLICATION
dep. on data-model
dep. on OS
dep. on GUI
comm comm
preloading
control
event
consumer
event
executor
logic
recording
control
playback
control
logic
data-model
adapter
user interface
preloading
action
EVENT DATA
EVENT DATA
GUI EVENTS
Open architecture divided into 2 elements:
HMI Tester → recording/playback control
Preload Module → deployed into the target application
→ event capture and execution
Adaptable modules + LIB Preload ⇔ Easy integration

How does it work?
OHT example video:

OHT Conclusions
basis to develop GUI testing tools based on GUI introspection
adaptable architecture + adaptable recording/execution processes
direct validation of system response → no model, no test oracles
we still need visual validation of results
Open-source implementation...
cross-platform (C++) + agnostic to GUI platform + extensible
...used in SAES → Qt and ILOG versions

Table of contents
4 Block I
5 Block II

Goal 1.2 & Problems Verifying User Input
→ Goal 1.2: Allow verification of user inputs at runtime
In Runtime Verification (RV) a program is monitored at
runtime to determine if it satisfies correctness properties
Problems of using RV to verify GUI input:
GUI properties are simpler → +effective method
complex languages and formal logic → +options
formal specification of acceptable behavior → lightweight method
aspect-oriented bindings are static → dynamic languages
feedback to users? → interactive verification

Solution: S-DAVER (Script-based Verification)
S-DAVER: lightweight framework to verify user inputs in GUI
→ verification layer between GUI and business logic
Based on fundamental pillars of RV...
GUI data verified when produced → runtime
efficient verification → while user interaction
...but:
RV verifies app behavior = S-DAVER verifies data
rules written in scripting languages → no formalisms
included in separate files
can be changed at runtime → dynamic approach
lightweight + easy-to-integrate → smooth verification process
rules loaded and verified transparently

verification
layer
business logic
1
2
rules logsrule loading
interaction
extraction
rule checking feedback
3
gui
dep. on rules
dep. on GUI
Independent/autonomous verification layer:
1 works while user interaction → no perceived overhead
2 only changed data is verified → efficiency
3 if rules fulfilled then go to business logic else do interactive feedback
Adaptable architecture ⇒ GUI platform + rules language

How does it work?
S-DAVER example video:
verification layer
rules logs
1 4
rule search rule check
2 3
Event: FocusLost
Widget: le_bufferSize
le_bufferSize change was invalid.
Rule: "le_bufferSize <= 20"

S-DAVER Conclusions
lightweight, easy-to-integrate verification solution
trust relationship GUI ⇔ business logic
interactive verification experience
no guarantee of the internal correctness of rules
cross-platform + agnostic to GUI platform and verification language
+ extensible
...tested in SAES → Qt + Lua version

Table of contents
4 Block I
5 Block II

Goal 2.1 & Problems Assessing Multimodal Interaction
→ Goal 2.1: Describe multimodal interaction for its assessment
Problems with multimodal interaction assessment:
many representations → abstract + generic
diﬀerent modalities analyzed separately → equivalences
static parameters or average metrics
dynamic nature of dialog process? → stepwise description
conversational nature of dialog? → interaction turns

Solution: PALADIN (Multimodal Interaction Model)
PALADIN: a generic, runtime multimodal interaction model
→ dynamic description of multimodal interaction
Dialog = sequence of alternative user/system turns
interaction described stepwise
relationship data⇔time → runtime model
allows dynamic analysis of interaction + cause-eﬀect relationships
system turn
user turn
ends
system
feedback
starts
info
presentation
starts
system turn
ends
user turn
ends
user starts
exploring
user starts
transferring
action
user turn
delay feedback presentation delay exploring transferring

Generic description of multimodal interaction:
model → relationships between data
model → same metrics + same structure → comparison
uniﬁed criteria to describe multimodal interaction
agnostic to system and modalities used

Model with 2 levels of abstraction:
generic metrics → main aspects of communication
speciﬁc metrics → peculiarities of modalities
Multimodal Interaction ⇒ described as a single ﬂow of actions

PALADIN Integration and Usage
1 PALADIN model instance = 1 interaction episode
Instances intended to be created automatically:
1 interaction extraction → automatic user-system instrumentation
2 PALADIN instantiation → helping framework
3 interaction analysis, comparison, transformation, and decision
PALADIN
instances
1. Interaction tracking 2. Model instantiation 3. Interaction analysis
INSTANTIATION
FRAMEWORK
HCI
EXTRACTOR
INTERACTION
ANALYZER
DECIDER
PALADIN
implementation
app

PALADIN Conclusions
generic description of multimodal interaction + stepwise
multimodal interaction = single, seamless flow of actions
unified criteria to analyze and compare different systems
model design + model implementation (Ecore + Java)
instrumentation tool + instantiation framework (Java + Android)
...tested with real users in TLabs
What about mobile interaction?
What about quality of experiences?

Table of contents
4 Block I
5 Block II

Main Problems when Assessing Mobile User Experiences
→ Goal 2.2: Allow analysis of user experiences in mobile scenarios
Problems analyzing mobile user experiences:
many interaction and context data → select useful ones
low standardization in context deﬁnition → generic context
incorporate user ratings into interaction analysis
no robust methodologies combining...
quantitative data evaluating performance
qualitative data assessing usability/QoE

Solution: CARIM (Context & Ratings Interaction Model)
CARIM = PALADIN + context data + users subjective ratings
combines mobile interaction data
runtime instrumentation of mobile interaction
ModalityTurnDialog
System TurnUser Turn
Metacommunication
Dynamic Context
Physical environment Social context Location/time Device Communication
Trial
User data
Ratings Mood
Attitude
PALADIN (base)
CARIM (new)

Incorporating Context Data
Generic deﬁnition of the
surrounding context:
parameter-based
6 dimensions
user is the subject
extensible
CARIM
deviceuser
location/time
communication social context
human-computer
interaction
dynamic
parameters
static
parameters
user ratings
environment
PALADIN

Incorporating User Ratings
User ratings = users quality perception + 2 influencing factors
Quality perception:
AttrakDiff questionnaire → pragmatic, hedonic, attractiveness
mini version (10 pairs of opposite adjectives)
Factors → high influence on quality perception
user mood →
attitude towards technology use → 5-item questionnaire

How does it work?
CARIM experiment scenario + video:

CARIM Conclusions
uniﬁed criteria to assess QoE from the perspectives of...
user-system interaction
its surrounding context
quality perception of the users
basis for context- and QoE-based decision
we are still asking too much data to users
model design + model implementation (Ecore + Java)
instrumentation tool + instantiation framework (Java + Android)
...tested in our labs with real users

III
Conclusions and Further Work

Table of contents
6 Conclusions
7 Future Lines of Work

Conclusions - Goals Achieved (i)
2 Approaches to assess interaction:
Block 1: Decompose interaction into its components
Block 2: Assess interaction as a whole
We outlined 4 main goals

Conclusions - Goals Achieved (ii)
→ Goal 1.1: Ease the validation of system response
expensive, hard-to-integrate testing processes
OHT ⇒ adaptable architecture for developing GUI testing tools
implement GUI testing on a robust basis
validate app response using a battery of test cases
test software with precision and tolerance to changes

Conclusions - Goals Achieved (iii)
→ Goal 1.2: Allow verification of user inputs at runtime
no effective GUI data verification + no feedback
S-DAVER ⇒ lightweight, runtime, interactive verification framework
independent layer to verify GUI data
dynamic rules
unified interactive feedback

Conclusions - Goals Achieved (iv)
→ Goal 2.1: Assess and compare multimodal interaction
static, no abstract description of modalities
PALADIN ⇒ generic, runtime, stepwise interaction model
1 model ⇔ 1 description ⇔ many scenarios
seamless description of modalities
runtime interaction model

Conclusions - Goals Achieved (v)
→ Goal 2.2: Allow analysis of mobile user experiences
no methods combining quantitative + qualitative
CARIM ⇒ runtime model = interaction + context + user ratings
measure and analyze QoE from diﬀerent perspectives
combined + structured data into model
generic deﬁnition of context

Table of contents
6 Conclusions
7 Future Lines of Work

Future Work (i)
Support automatic validation of GUI output
OHT ⇒ still need visual validation of results
graphical computing → simulate visual validation
Ensure correctness and consistency of speciﬁcations
S-DAVER ⇒ no method to guarantee rules correctness
explore alternatives → CLP?
Outline Block 1 concepts into a multimodal context
from only-GUI to multimodal scenario
validation of multimodal input/output

Future Work (ii)
Enhance/extend design of interaction models
PALADIN & CARIM ⇒ limitations in design
ﬁx potential errors/new emerging needs
instrumentation of new modalities → gestures
incorporate new dimensions of interaction
task: interaction issues ⇔ problem being solved
user emotions: interaction issues ⇔ user state

Future Work (iii)
Exploit the full potential of the interaction models
PALADIN & CARIM ⇒ structured data
model transformation/code generation
decision systems based on live data → adaptation
continuous authentication of users

Future Work (iv)
Study interaction in multi-person, concurrent interfaces
multiple users interacting with the system concurrently
e.g., smart home, tabletop devices
reconsider turn-based approach and interaction models

Table of contents
8 Scientiﬁc Publications
9 Software Contributions

JCR Journals
1 Pedro Luis Mateo Navarro, Gregorio Mart´ınez Pérez, Diego Sevilla
Ruiz. OpenHMI-Tester: An Open and Cross-Platform
Architecture for GUI Testing and Certification, International
Journal of Computer Systems Science and Engineering (IJCSSE),
Special Issue on Open Source Certification, 2010.
2 Pedro Luis Mateo Navarro, Stefan Hillmann, Sebastian Möller, Diego
Sevilla Ruiz, Gregorio Mart´ınez Pérez. Runtime Model Based
Framework for Automatic Evaluation of Multimodal Interfaces,
Journal on Multimodal User Interfaces (JMUI), 2014. In press.
Ruiz. A Context-aware Interaction Model for the Analysis of
Users QoE in Mobile Environments, International Journal of
Human-Computer Interaction (IJHC), Taylor & Francis, 2014. In
press.

JCR Journals under Review
4 Pedro Luis Mateo Navarro, Diego Sevilla Ruiz, Gregorio Mart´ınez
Pérez. A Lightweight Framework for Dynamic GUI Data
Verification Based on Scripts, Journal on Software Testing,
Verification and Reliability, 2014. Minor revision.
Solution JCR journal
OHT
S-DAVER minor revision
PALADIN
CARIM

Other Journals
Ruiz. Aplicación de Open HMI Tester como framework
open-source para herramientas de pruebas de software, Revista
Española de Innovación, Calidad e Ingenier´ıa del Software (REICIS),
2009.
Pérez. A Proposal for Automatic Testing of GUIs based on
Annotated Use Cases, Advances in Software Engineering journal,
Special Issue on Software Test Automation, vol. 2010, 2010.
Pérez. CARIM: Un Modelo de Interacción para el Análisis de la
QoE en Entornos Móviles, FAZ: Revista de Diseño de Interacción,
2014. In press.

Conferences and Workshops (i)
Pérez. A Context-aware Model for QoE Analysis in Mobile
Environments, XIV International Congress on Human-Computer
Interaction, Madrid (Spain), September 2013.
Pérez. A Context-aware Model for the Analysis of User
Interaction and QoE in Mobile Environments, CENTRIC 2012,
Lisbon (Portugal), November 2012.
3 Pedro Luis Mateo Navarro, Stefan Schmidt. Model-based
Measurement of Human-Computer Interaction in Mobile
Multimodal Environments, aMMI (Assessing Multimodal
Interaction), NordiCHI 2012, Copenhagen (Denmark), October 2012.
Pérez. Towards Software Quality and User Satisfaction through
User Interfaces, Fourth IEEE International Conference on Software
Testing, Berlin (Germany), March 2011.

Conferences and Workshops (ii)
Ruiz. Verificación de Datos en la GUI como un Aspecto
Separado de las Aplicaciones, PRIS 2010, Jornadas de Ingenieria
del Software y Bases de Datos (JISBD 2010), Valencia (Spain),
September 2010.
Ruiz. Open HMI Tester: un Framework Open-source para
Herramientas de Pruebas de Software, PRIS 2009, Jornadas de
Ingenieria del Software y Bases de Datos (JISBD 2009), San
Sebastian (Spain), September 2009.
Pérez. Automated GUI Testing Validation guided by Annotated
Use Cases, MOTES 09 - Model-based Testing Workshop, Lübeck
(Germany), September 2009.

Table of contents
8 Scientiﬁc Publications
9 Software Contributions

Main Software Contributions
OHT
Testing framework (C++, Qt4)
http://www.catedrasaes.org/wiki/OHT
S-DAVER
Veriﬁcation framework (C++, Qt4) + Rules (Lua/Chaiscript)
http://www.catedrasaes.org/wiki/GuiVerification
PALADIN
Interaction model (EMF, Java) + HCI Extractor 8 (Java, Android)
http://www.catedrasaes.org/wiki/MIM
CARIM:
Interaction model (EMF, Java) + HCI Extractor 11 (Java, Android)
http://www.catedrasaes.org/wiki/Carim

Main Software Contributions — Validation
Tools validated into diﬀerent scenarios:
OHT SAES developments
S-DAVER and internal processes
PALADIN experiments in TLabs
CARIM experiments in C´atedra-SAES lab
All tools are open-source, downloadable,
and ready-to-use

Acknowledgements
www.catedrasaes.org
Supported by:
cátedrasaeslabs

PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces

Ähnlich wie PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

PhD Thesis Defense - Enhancing Software Quality and Quality of Experience through User Interfaces