The presentation of our Journal First (J1ST) paper entitled "Conceptualisation, measurement, and application of semantic transparency in visual notations" on MODELS 2021: ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS)

1. Conceptualization, measurement, and application of
semantic transparency
in visual notations
Journal First
Presented by Gregor Polančič

2. Kuhar, S., Polančič, G. Conceptualization, measurement, and application of semantic transparency in
visual notations. Softw Syst Model (2021). https://doi.org/10.1007/s10270-021-00888-9
Received: 3 September 2020 / Revised: 12 April 2021 / Accepted: 16 April 2021

3. Visual communication is the most effective way of
passing information because the human mind
processes things in images.

4. Digrammatic communication
• Visual communication in form of diagrams has a crucial role in the planning, analysis,
documentation, and communication of information systems and related concepts.
UML Class diagram UML Use cases diagram BPMN Choreography diagram
BPMN Choreography diagram
IT Infrastructure modelling in UML
ER diagram
UML Sequence diagram
Archimate diagram
[van der Linden, D., Hadar, I., Zamansky, A.: What practitioners really want: requirements for visual notations in
conceptual modeling. Softw. Syst. Model. 18(3), 1813–1831 (2019). https://doi. org/10.1007/s10270-018-0667-4]

5. Diagrams are based on notations
5
Notation Vocabulary Grammar Semantics
„Instance of“
Visual
diagram
Set of all
graphical signs
Rules for
constructing
valid diagrams
The meaning of
(combinations)
signs
[Moody, D.: The physics of notations: toward a scientific basis for constructing visual notations in software
engineering. IEEE Trans. Softw. Eng. 35(6), 756–779 (2009). https://doi.org/10.1109/TSE. 2009.67]

6. Notations must meet specific requirements to
achieve high levels of cognitive effectiveness
e.g., analyst e.g., process
participant
Interpreted
diagram
Interpreted
diagram
Level of matching (effectiveness)
The visual vocabulary plays a crucial role in the effectiveness of diagrams since the visual
form has an equal or even greater influence on cognitive load than the content.
[Larkin, J., Simon, H.: Why a diagram is (sometimes) worth ten thousand words. Cogn. Sci. 11(1), 65–100 (1987). https://doi.org/ 10.1016/S0364-0213(87)80026-5]

7. Semantic transparency
• “The key to designing visual notations that are understandable to naïve users is a property
called semantic transparency” (Caire et al.,2013), which means that the meaning (semantics) of a
sign is clear (i.e., intuitive, transparent) from its appearance alone.
• Introduced by Moody and Hillegersberg in 2008 as perceptual immediacy.
Caire, P., Genon, N., Heymans, P., & Moody, D. L. (2013). Visual notation design 2.0: Towards user comprehensible
requirements engineering notations. In 2013 21st IEEE International Requirements Engineering Conference, RE 2013 -
Proceedings (pp. 115-124). [6636711] https://doi.org/10.1109/RE.2013.6636711

8. Moody‘s Physics of Notations (PoN)
• Currently, the most popular and comprehensive recommendations for assuring
cognitively effective notations.
• PoN principles: semiotic clarity, graphic economy, visual expressiveness, dual
coding, manageable complexity, cognitive integration, cognitive fit, perceptual
discriminability, semantic transparency.
• The semantic transparency principle states that semantically transparent representations
reduce the cognitive load because they have built-in mnemonics.
• PoN has been many times criticized for lacking operationalization of the
principles and their systematic validation or replication.
• → The investigation of semantic transparency varies among studies in aspects of
the definition, metrics, research methods, collection instruments, and measured
data types.
[Moody, D.: The physics of notations: toward a scientific basis for constructing visual notations in software engineering. IEEETrans. Softw. Eng. 35(6), 756–779 (2009).
https://doi.org/10.1109/TSE.2009.67]

9. Research goals
1. To comprehensively define ST including its operationalization,
2. To identify and summarize existing ST applications in evaluation and design of visual notations,
and
3. To identify any gaps in current research in order to propose future investigations in this field.
Types of visual signs and degrees of semantic transparency

10. Research questions
• RQ1: How is semantic transparency defined?
• RQ1.1: How is semantic transparency defined across the studies and disciplines?
• RQ1.2: Which theories are related to semantic transparency?
• RQ1.3: Which synonyms are related to semantic transparency?
• RQ1.4: Which concepts are related to semantic transparency?
• RQ2: How is semantic transparency evaluated?
• RQ2.1: How is semantic transparency measured?
• RQ2.2: Which research methods are applied in the evaluation of semantic transparency?
• RQ3: How is semantic transparency applied?
• RQ3.1: How is the application of semantic transparency approached when designing new notations?
• RQ3.2: Does the introduction of a new notation cover the complete sets of semantic and visual concepts?
• RQ3.3: Is the design of new signs rationalized?

11. Research method: systematic
literature review
(“semantic transparency” OR “perceptual directness” OR “perceptual immediacy” OR “perceptual
transparency”) AND (modeling OR language OR notation OR visual OR diagram)
@
SpringerLink, ACM DL, IEEE Xplore, Science Direct, and ISI Web of Science.

12. Inclusion and exclusion criteria
EC1: The study does not relate to ST, as
described in PoN.
EC2: The study does not relate to visual
notations.
EC3: The study does not evaluate an
existing notation or define a new notation
in relation to ST.
IC1: The study must be published after
2008 (as first appeared in the context of
visual notations).
IC2: The study is a primary study.

13. Selection process

14. List of selected studies (N=94)

15. Results
i.e., answers to research questions

16. The definition …

17. RQ1.1: How is semantic transparency defined
across the studies and disciplines?
• 72/74 studies that defined ST declared ST is achieved when visual signs imply their meaning.
• 9/72 studies defined that ST is achieved when icons are used.
• 6/72 and six studies defined ST is achieved when certain spatial arrangements are used.
Icons in visual notations do not
guarantee high ST, as icons can
be transparent, opaque,
translucent, or even perverse.
Moody, D.: The physics of notations: toward a scientific
basis for constructing visual notations in software
engineering. IEEETrans. Softw. Eng. 35(6), 756–779 (2009).
https://doi.org/10.1109/TSE. 2009.67
Semantically transparent
spatial arrangements
can be challenging, especially
in complex diagrams
with many interrelated
elements.
21% of studies did not define ST,
even though they used this
principle either for the evaluation
or for the design of new notation.

18. RQ1.2:Which „theories“ are related to semantic
transparency?
57% of the studies defining
ST, used only PoN [113] for
the reference.
Masri et al. was referenced
mainly for the definition as
well as the related
concepts of ST, such as
understandability.
The most referenced work
for ST’s operationalization
is ISO 9186 [78].
None of the studies
referenced PoN for the
operationalization.
Petre [127] was mostly
referenced for the related
concept cognitive load.

19. RQ1.3 Which synonyms are related to semantic
transparency?
• ST is a well-accepted term as only 13% of the
studies listed synonyms.
• The most frequent ST’s synonyms are
naturalness, perceptual immediacy, semantic
distance, perceptual directness, and visual
onomatopoeia.
• Familiarity is not a synonym of ST, but it
influences sign recognition (mediating
concept).
Synonyms Studies
Familiarity [99,137]
Naturalness [64,117]
Perceptual immediacy [12,114]
Semantic distance [34,50]
Visual/perceptual directness [55,116]
Visual onomatopoeia [24,56]
Closeness of mapping [111]
Intuitiveness [17]
Pragmatic quality [17]
Readability [17]

20. RQ1.4:Which concepts are related to semantic
transparency?
• Dependent concepts, where the relation to ST was empirically
demonstrated are cognitive load, learnability, and usability.
• Dependent concept, where the relation to ST was not empirically
demonstrated is usefulness.
• Cognitive effectiveness was not entirely demonstrated to be related
to ST, as only four out of eleven variables were confirmed to be
related.
• Moderating concepts that were found to be related to ST
were design rationale, cultural associations, familiarity,
and aesthetics.
• Cognitive load was listed most often as a dependent
concept related to ST.
Empirically
demonstrated
factors and
relatinships
only.

21. RQ1: How is semantic transparency defined?
• Aspects of semantic transparency definition:
• ST is achieved when signs have a visual form that implies their meaning.
• ST is achieved when icons are used.
• ST is achieved when certain spatial arrangements are used.
• Familiarity must not be mistaken with ST as it is a concept related to users’ previous knowledge,
rather than to the sign’s characteristics or spatial arrangements.
Not entirely rational since
graphical icons can have any
degree of ST, including semantic
perversity.
If icons are comprehended in semiotic terms, then
the first (signs imply their meaning) and the second
(icons are used) ST definition aspects describe the
same feature of ST.
→
The semiotic definition of an icon is “use of signs
that imply their meaning.”

22. The measurement …

23. RQ2.1: How is semantic transparency
measured?
• Of 94 researched studies, 64 (68%) were
describing the evaluation of ST.
• Three studies were describing two
evaluations, and one study three evaluations.
• Thus, 69 evaluations were counted altogether.
• In 51% of studies that were evaluating ST, no
metrics were used.
• The majority of the studies (17%) used the
metric hit rate with recognition test, where
participants had to match the sign with the
right concept.
• All metrics are defined with user test as a
method of application, except one, namely
expert evaluation.

24. Catalogue of metrics based on ISO/IEC 9126-4: Software
Engineering—Software product quality—Part 4: Quality in use metrics

25. Catalogue of metrics based on ISO/IEC 9126-4: Software
Engineering—Software product quality—Part 4: Quality in use metrics

26. RQ2.2:Which research methods are applied in the
evaluation of semantic transparency?
• Most of the 69 evaluations used either survey or expert analysis for the
evaluation method.
• In the surveys, the questionnaires were used most often as a collection
instrument, whereas in expert analysis, the information on collection
instruments was rarely provided.
• By profession, the evaluators were mostly students or domain-specific
experts.
• Users were mostly included in the surveys, and article authors were
mostly performing expert analysis.
• None of the experts reported on using a protocol for evaluation.
• Users who were involved in the evaluations were mostly novices (45%),
not having previous knowledge in the researched notation.

27. Applied research methods and instruments
Empirical methods in
relation to data
collection instruments.
(N=69)

28. Applied question types and data types
Question types in
questionnaires.
(N=69)
Data types in relation to
empirical methods.
(N=69)

29. Evaluator characteristics
Evaluator types in relation
to their profession.
(N=69)
Evaluator types in relation
to empirical methods.
(N=69)

30. Samples
Box plot diagram presenting the
distribution of the number of users
in evaluations.
(N=29)
Bar chart presenting
users’ expertise.
(N=29)

31. RQ2.2:Which research methods are applied in the
evaluation of semantic transparency?

32. The application …

33. RQ3.1: How is the application of semantic transparency approached when designing
new notations?
RQ3.2: Does the introduction of a new notation cover the complete sets of semantic
and visual concepts?
• Search for explicit arguments on how ST was used for designing new notations in the studies.
• 39 studies (41%) were found that applied ST while designing new notations or improving old ones.
Percentage of ST
application approaches.
(N=39)
Percentage of studies where
complete („1:1“) or partial sets
of concepts were visually and
semantically presented.
(N=39)

34. RQ3.3: Is the design of new signs rationalized?
• A design rationale was considered as an
explicit argument on why a visual sign
was selected for a particular concept.
• Only 3 studies provided a design
rationale for all newly designed signs.
• The studies with only a few new signs
(1,2, and 4).
The distribution of the
numbers of signs
visually presented.
(N=39)
The number of signs for
which a design rationale
was provided.
(N=39)
Mean number of signs
visually presented.
(N=39)

35. RQ3: How is semantic transparency applied?
• The most common approaches provided for ST
application in the design of new notations were
including signs, which imply their meaning (44%)
and using icons (28%).
• The majority (59%) of the design studies presented
complete sets of concepts visually and semantically.
• Only three (8%) studies provided a design rationale
for all newly designed signs.
• Design rationale and familiarity can positively
impact the recognition of signs.
https://www.omg.org/spec/CMMN
https://www.omg.org/spec/BPMN

36. Conclusions and recommendations
• Graphical icons can have any level of ST, including semantic
perversity.
• Researchers are therefore advised against ST presence presumptions
because of icon usage and are recommended to conduct empirical tests
when evaluating ST.
• ST impacts cognitive effectiveness only partially.
• Cognitive load, a concept measured with a perceived
cognitive load variable in the form of surveys, has been
demonstrated to be related to ST.
ST
Cognitive
effectiveness
ST
Cognitive
load

37. Conclusions and recommendations
• To diminish the influence of moderating concepts, empirical tests
should be planned with the inclusion of participants of different
ages.
• In the process of designing new notations, moderating concepts should be either
embraced (design rationale and familiarity) or diminished (cultural associations).
• To lower the guessing occurrence, comprehension tests should be
performed using open-type questions or both open and closed-type
questions.
• In expert studies, the verification level was generally low as none of
the evaluators followed any evaluation protocol, the evaluators were
in larger share article authors, and argumentation on ST grading was
seldom provided.
+
ST
Concrete
syntax
Design rationale,
cultural associations,
familiarity, aesthetics

38. Future work
• Our further research will focus on improving the methods and techniques for
evaluating and applying ST.
ST
Concrete
syntax
Evaluating ST
Applying ST

39. Thank you for your attetion!
gregor.polancic@um.si
Kuhar, S., Polančič, G.
Conceptualization, measurement, and application of semantic transparency in visual notations.
Softw Syst Model (2021).
https://doi.org/10.1007/s10270-021-00888-9
www.polancic.com