The document summarizes Vlad Acrețoaie's PhD thesis on developing model manipulation languages for end-user modelers. It introduces the Visual Model Query Language (VMQL), Visual Model Constraint Language (VMCL), and Visual Model Transformation Language (VMTL) designed to be more usable for non-programmers. Experiments showed VMQL and VMTL had good learnability and usability. Future work includes improving tool support, more qualitative evaluations, and building a theory of learnability for model manipulation languages. The thesis takes initial steps to develop highly usable languages for end-user modelers.

1. Model Manipulation for End-User Modelers
Student:
Vlad Acrețoaie (Technical University of Denmark)
Supervisor:
Assoc. Prof. Harald Störrle (Technical University of Denmark)
Assessment committee:
Assoc. Prof. Ekkart Kindler (Technical University of Denmark), Chair
Prof. Dr. Michel Chaudron (University of Gothenburg, Chalmers University of Technology)
Prof. Dr. Gabriele Taentzer (Philipps-Universität Marburg)
Defense chairman:
Assoc. Prof. Hubert Baumeister (Technical University of Denmark)
PhD Defense – Technical University of Denmark – 12.05.2016

2. Model Manipulation
for End-User Modelers
2
Scenario: modeling in the financial domain

3. Model Manipulation
for End-User Modelers
3
Modeling is …
“the cost-effective use of something
in place of something else
for some cognitive purpose” [1]
[1] Jeff Rothenberg. The Nature of Modeling. RAND Note N-3027-DARPA, The RAND Corporation, November 1989.

4. Model Manipulation
for End-User Modelers
4
Model examples

5. Model Manipulation
for End-User Modelers
5
Model manipulation
Model querying
 Retrieves all model elements or model fragments included in a
model that satisfy a given condition.
Model constraints
 Retrieve all model elements or model fragments included in a
model that violate a given condition.
Model transformation
 Produces different models, viewpoints, or artifacts based on a
source model and a transformation specification.

6. Model Manipulation
for End-User Modelers
6
Model transformation
Target metamodel
Transformation
specification
Source model Target model
Transformation
engine
Reads Writes
Conforms to Conforms to
Refers to Refers to
Executes
Source metamodel
or other artifacts
(code, documentation)
model transformation language (MTL)
Expressed using
Adapted from: Krzysztof Czarnecki and Simon Helsen. Feature-Based Survey of Model Transformation Approaches. IBM
Systems Journal, 45(3):621–645, 2006.

7. Model Manipulation
for End-User Modelers
7
Example transformation specification
The Epsilon Transformation Language (ETL)

8. Model Manipulation
for End-User Modelers
8
Model-Driven Engineering (MDE)
 A software development methodology.
 Models replace code as the principal software development
artifact: “everything is a model”.
 All other artifacts (source code, test cases, documentation) are
generated from sufficiently detailed models.
 Model transformation languages allow developers to specify how
each of these artifacts, as well as increasingly detailed models,
should be generated from the existing models.

9. Model Manipulation
for End-User Modelers
9
Different kinds of modelers
Domain Modeling Meta-
modeling
Model
manipulation
Programming
MDE practitioner     
Language engineer     
End-user modeler     
Definition
End-user modelers are non-programmer users of a modeling
language familiar with its concrete syntax and semantics, but
unfamiliar with its metamodel, abstract syntax, and applicable
model manipulation languages.

10. Model Manipulation
for End-User Modelers
10
Human factors in model transformation
 RQ1 To which extent are human factors addressed in model
transformation research?
 RQ2 Which existing model transformation languages address
human factors concerns?
 RQ3 What is the quality of the evaluations carried out in human
factors studies in the model transformation area?
“… understanding how knowledge of human strengths and
limitations, both mental and physical, can lead to better
system design, more effective training of the user, and better
assessment of the usability of a system” [2]
[2] Christopher D. Wickens, John D Lee, Yili Liu, and Sallie Gordon-Becker. Introduction to Human Factors Engineering.
Pearson, second edition, 2003.

11. Model Manipulation
for End-User Modelers
11
Systematic Mapping Study
 These research questions were addressed via a Systematic
Mapping Study (SMS) [3] investigating the usability and
maintainability of model transformation approaches.
[3] Barbara A. Kitchenham and Stuart Charters. Guidelines for performing Systematic Literature Reviews in Software
Engineering. EBSE Technical Report EBSE-2007-01, Keele University, University of Durham, July 2007.
Usability attributes
• Learnability
• Readability
• Understandability
• Comprehensibility
• Aesthetics
• User satisfaction
• User comfort
Maintainability attributes
• Modularity
• Composition support
• Reusability
• Modifiability
• Testability
• Debugging support
• Traceability

12. Model Manipulation
for End-User Modelers
12
Included studies

13. Model Manipulation
for End-User Modelers
13
Research topics
Unaddressed usability topics:
• comprehensibility
• aesthetics
• user satisfaction
• user comfort

14. Model Manipulation
for End-User Modelers
14
Study types

15. Model Manipulation
for End-User Modelers
15
Addressed transformation languages

16. Model Manipulation
for End-User Modelers
16
Addressed modeling technologies

17. Model Manipulation
for End-User Modelers
17
Implementation languages

18. Model Manipulation
for End-User Modelers
18
Evaluation methods

19. Model Manipulation
for End-User Modelers
19
Quality of empirical evaluations
Checklist Item Affirmative answers
All studies
Rationale statement 36 of 47 (77%)
Context described 31 of 47 (66%)
Data collection methods described 12 of 47 (26%)
Outcomes presented 42 of 47 (89%)
Threats to validity discussed 15 of 47 (32%)
Featuring human participants 9 of 47 (19%)
Participants appropriately described 6 of 9 (67%)
Quantitative studies
Small sample size (< 10) 16 of 26 (62%)
Using a control group 7 of 26 (27%)
Analysis methods justified 4 of 26 (15%)
Qualitative studies
Addressing original aims 16 of 23 (70%)
Generalizing conclusions 4 of 23 (17%)

20. Model Manipulation
for End-User Modelers
20
A vision for end-user modelers

21. Model Manipulation
for End-User Modelers
21
The VM* language family
 The Visual Model Query Language (VMQL)
 Follows the query by-example paradigm
 The Visual Model Constraint Language (VMCL)
 Is a constraint-oriented extension of VMQL
 The Visual Model Transformation Language (VMTL)
 Model-to-model transformations
 Both in-place and out-place transformations
 Endogenous transformation

22. Model Manipulation
for End-User Modelers
22
Example: a loans domain model

23. Model Manipulation
for End-User Modelers
23
Example: a loans domain model

24. Model Manipulation
for End-User Modelers
24
Example VMQL queries (1/3)
“Which Use Cases can a customer perform?”

25. Model Manipulation
for End-User Modelers
25
Example VMQL queries (2/3)
“Find all abstract Classes.”

26. Model Manipulation
for End-User Modelers
26
Example VMQL queries (3/3)
“Which Classes are associated to the ‘Customer’ Class?”

27. Model Manipulation
for End-User Modelers
27
VMCL constraint example
“Loan details must be specified for all loan types other than revolving loans.”

28. Model Manipulation
for End-User Modelers
28
VMCL constraint example

29. Model Manipulation
for End-User Modelers
29
Example VMTL transformations (1/2)

30. Model Manipulation
for End-User Modelers
30
Example VMTL transformations (2/2)

31. Model Manipulation
for End-User Modelers
31
Transparent Model Manipulation
Syntax
transparency
Environment
transparency
Execution
transparency
A language capable of expressing model
manipulations on source models conforming to
metamodel M and (optionally) producing target
models also conforming to metamodel M is
syntax transparent with respect to M iff all such
specifications conform to M.
A model manipulation language is environment
transparent iff it allows users to adopt their
preferred editor for creating specifications and
viewing the results of applying these
specifications.
A model manipulation language is execution
transparent iff manipulations expressed using it
can be executed by several execution engines.

32. Model Manipulation
for End-User Modelers
32
Achieving syntax transparency
Host language prerequisites:
• Availability of a scoping construct (e.g. packages)
• Availability of textual annotations
• Availability of a profiling mechanism (can be replaced by naming conventions)
Transformation specifications are valid instances of their respective host language.

33. Model Manipulation
for End-User Modelers
33
Achieving environment transparency
Users can adopt their preferred editor for each transformation artifact.

34. Model Manipulation
for End-User Modelers
34
Achieving execution transparency
Users can select their preferred model transformation execution engine.

35. Model Manipulation
for End-User Modelers
35
VMQL learnability experiment
Methods and materials
Setup
• within subjects design
• paper forms
• each participant answers all questions
• BPMN process diagrams
Population
24 undergraduate Computer Science
students with homogenous levels of
experience in modeling, BPMN, and
model querying languages

36. Model Manipulation
for End-User Modelers
36
VMQL learnability experiment
Results: comprehension and cognitive load
Task scores Cognitive load

37. Model Manipulation
for End-User Modelers
37
VMTL learnability experiments
Methods and materials

38. Model Manipulation
for End-User Modelers
38
VMTL learnability experiments
Results: comprehension

39. Model Manipulation
for End-User Modelers
39
VMTL learnability experiments
Results: completion time

40. Model Manipulation
for End-User Modelers
40
VMTL learnability experiments
Results: required effort

41. Model Manipulation
for End-User Modelers
41
Think-aloud protocol analysis
VMTL learnability
Participant Occupation Domain UML [1..5] English [1..5]
1 PhD student Nutrition Science 1 4
2 PhD student Theoretical Computer Science 1 4
3 PhD student Software Engineering 2 2
4 PhD student Software Engineering 2 3

42. Model Manipulation
for End-User Modelers
42
Think-aloud protocol analysis
Results
 The background of individual users plays an important role in their
comprehension capabilities.
 Confirmed design decisions
 Offering both Update Patterns and Find/Replace Patterns.
 Making the use of pattern icons optional.
 Problematic aspects:
 Identifying annotation anchor points.
 Interpreting annotations as “log messages”.
 Confusion created by the create singleton annotation, subsequently
replaced by create if not exists.

43. Model Manipulation
for End-User Modelers
43
Summary: achieved objectives
 Defining and characterizing end-user modelers as a distinct category
of model manipulation language users.
 Substantiating the claim that existing model transformation
languages and tools do not emphasize human factors considerations
that are important to end-user modelers.
 Proposing a coherent family of model manipulation languages and
tools intended to meet the requirements of end-user modelers.
 Demonstrating that the provided languages meet some end-user
modeler requirements to a greater extent than existing languages.

44. Model Manipulation
for End-User Modelers
44
Summary: limitations
 In its existing form, VMTL does not support exogenous
transformations. There is an inherent conflict between syntax
transparency and exogenous specifications.
 Only one of the presented learnability experiments features a
production task, and all include participants that are only partially
representative for the general population of end-user modelers.
 The implementation of tool support for the VM* languages is
currently incomplete.
 A performace evaluation of the VM* tool support is lacking, but
execution transparency likely brings a performance penalty.

45. Model Manipulation
for End-User Modelers
45
Future work
 Tool support improvements
 Full support for current specifications
 Alternative execution engines
 Implementing model editor plugins (e.g. MagicDraw, Papyrus, Visio)
 Why are the VM* languages easy/difficult to learn?
 More qualitative evidence: interviews, think-aloud protocol analyses
 Including production tasks and end-user modelers as participants
 Ultimate goal: building a theory of learnability/usability for model
manipulation languages in the end-user modeler context
 As in the case of general-purpose programming languages, there is
a long path towards highly usable model manipulation languages.
 This thesis is just one of the first steps in this direction.

46. Model Manipulation for End-User Modelers
- Additioal Slides -
PhD Defense – Technical University of Denmark – 12.05.2016
Student:
Vlad Acrețoaie (Technical University of Denmark)
Supervisor:
Assoc. Prof. Harald Störrle (Technical University of Denmark)
Assessment committee:
Assoc. Prof. Ekkart Kindler (Technical University of Denmark), Chair
Prof. Dr. Michel Chaudron (University of Gothenburg, Chalmers University of Technology)
Prof. Dr. Gabriele Taentzer (Philipps-Universität Marburg)
Defense chairman:
Assoc. Prof. Hubert Baumeister (Technical University of Denmark)

47. Model Manipulation
for End-User Modelers
47
Models, originals, and modelers

48. Model Manipulation
for End-User Modelers
48
Publication venues

49. Model Manipulation
for End-User Modelers
49
Studies by publication year

50. Model Manipulation
for End-User Modelers
50
Empirical evaluation quality scores
Quantitative
studies
Qualitative
studies

51. Model Manipulation
for End-User Modelers
51
The structure of a VMQL query

52. Model Manipulation
for End-User Modelers
52
The structure of a VMCL constraint

53. Model Manipulation
for End-User Modelers
53
The structure of a VMTL transformation

54. Model Manipulation
for End-User Modelers
54
Achieving environment transparency
Users can adopt their preferred editor for each transformation artifact.

55. Model Manipulation
for End-User Modelers
55
The VM* API
An example HTTP session

56. Model Manipulation
for End-User Modelers
56
Execution transparency in VMQL
Users can select their preferred model transformation execution engine.

57. Model Manipulation
for End-User Modelers
57
Execution transparency in VMCL
Users can select their preferred model transformation execution engine.

58. Model Manipulation
for End-User Modelers
58
Example model transformation
Removing an Activity Diagram anti-pattern

59. Model Manipulation
for End-User Modelers
59
Example model transformation
Specification in Epsilon

60. Model Manipulation
for End-User Modelers
60
Example model transformation
Specification in Henshin

61. Model Manipulation
for End-User Modelers
61
Example model transformation
Specification in VMTL

62. Model Manipulation
for End-User Modelers
62
VMTL learnability experiments
Results: task difficulty

63. Model Manipulation
for End-User Modelers
63
Execution engine requirements
 A rule-like construct
 Self-contained execution units that VMTL rules can be mapped to.
 Pattern matching
 A mechanism for expressing model patterns and matching them on models
 Variables
 Statically or dynamically typed variables with user-defined names and values
 Optional: rule scheduling
 This can be implemented on top of the execution engine

64. Model Manipulation
for End-User Modelers
64
Summary: lessons learned
 Usability, expressiveness, and predictability
 Increased expressiveness usually detracts from usability.
 Increased usability can negatively impact predictability.
 Pitfalls of model transformation systematic literature reviews
 There is a very large number of publications in this area.
 Many publications misuse empirical software engineering terminology.
 Experimental evaluation
 Evaluating transformation languages raises additional problems compared to
evaluating query and constraint languages.
 The evaluation methods adopted for general-purpose programming
languages could be beneficial for model transformation languages.