Slides of my presentation at BPM Cluster meeting, 21 August 2014, Eindhoven, NL

1. 1/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
Jan Claes
Teaching assistant : PhD 2009 – 2015
Supervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Why do people struggle with the complexity
of constructing a process model?

2. 2/31
www.janclaes.info
Business Process Modeling (BPM)
Assumptions
 Constructing process models is useful
 Constructing the models is cognitively challenging
Business Process Modeling (BPM)
 Process model quality
“what is a good process model?”
 Process of process modeling (PPM)
“how to make a good process model (effectiveness)
using a good approach (efficiency)?”

3. 3/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
How do people construct process models?

4. 4/31
www.janclaes.info
PPMChart visualization
 CREATE_ACTIVITY
 CREATE_START_EVENT
 CREATE_END_EVENT
 CREATE_AND
 CREATE_XOR
 CREATE_EDGE
 MOVE_ACTIVITY
 MOVE_START_EVENT
 MOVE_END_EVENT
 MOVE_AND
 MOVE_XOR
 DELETE_ACTIVITY
 DELETE_START_EVENT
 DELETE-END_EVENT
 DELETE_AND
 DELETE_XOR
 DELETE_EDGE
 NAME_ACTIVITY
 RENAME_ACTIVITY
 NAME_EDGE
 RENAME_EDGE
 Start event
 Edge
 Activity
 Gateway
 Edge
 Activity
 Edge
 Edge
 Activity
 Edge
 Gateway
 Edge
1
2
3
5
7
9
4
6
8
11
12
10
time
model
elements

5. 5/31
www.janclaes.info
Flow-oriented modeling

6. 6/31
www.janclaes.info
Aspect-oriented modeling

7. 7/31
www.janclaes.info
Undirected modeling

8. 8/31
www.janclaes.info
Observations (objective)
Obs. 1. Majority used serialized approach
Obs. 2. Large group flow-oriented modeling
Obs. 3. Small group aspect-oriented modeling
Obs. 4. Large group combine FO and AO
Obs. 5. Small group undirected modeling
Obs. 6. Remainder is uncategorized
Obs. 7. Undirected sessions lasted longer

9. 9/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
What about the quality
of the corresponding models?

10. 10/31
www.janclaes.info
Impressions (subjective)
Impr. 1. Modelers need serialization
Impr. 2. Structured serialization
helps avoiding ‘mistakes’
Impr. 3. Structured serialization
does not support every modeler
to the same extent

11. 11/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
Is there a rational for these
observations and impressions?

12. 12/31
www.janclaes.info
Kinds of human memory
input
Sensory
memory
Working
memory
Long term
memory
performance
OBSERVATION SELECTION
ORGANIZATION
RETRIEVAL
TRANSFER
Working
memory

13. 13/31
www.janclaes.info
Types of cognitive load
Time
Instantaneous
load
overall load
intrinsic
load
extraneous
load
germane
load
free
capacity
assumed capacity
instantaneous
load
overload

14. 14/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–

15. 15/31
www.janclaes.info
Cognitive Fit Theory (CFT)
Need for structure
Learning style
Field dependency
Sensory
Visual
Inductive
Active
Sequential
Intuitive
Auditory
Deductive
Reflective
Global
I ♥︎ chaos I #$@&! chaos
DON’T SEE THE FOREST THROUGH THE TREES

16. 16/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
What can we do with this knowledge?

17. 17/31
www.janclaes.info
Impressions
Cognitive theory
Observations
Theory building
Structured Process
Modeling Theory
17/32
www.janclaes.info

18. 18/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
1 2 3

19. 19/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
How will the developed theory be tested?

20. 20/31
www.janclaes.info
Methodology
1. Cognitive tests prior to the test session
 Collect modeler data
2. Observational modeling session
 At TU/e in BPM lecture of September 26
 Collect modeler, modeling and model data
3. Post-experiment interviews
with selection of participants (e.g., outliers)
 Collect additional, qualitative data

21. 21/31
www.janclaes.info
Experiment set-up
No treatment
Measure every involved construct of the theory
 Modeler, modeling process, process model
Calculate (cor)relations of the theory

22. 22/31
www.janclaes.info
PRIOR TO THE EXPERIMENT SESSION

23. 23/31
www.janclaes.info
Experiment set-up
1. Prior to the experiment session
 Software to be installed on personal laptop
( = download from website and unzip)
 5 cognitive tests (time range 5-30 min)
• Reading span test (RSPAN)
• Operation span test (OSPAN)
• Counting span test (CSPAN)
• Hidden figures test (HFT)
• Need for structure (NFS)

24. 24/31
www.janclaes.info
Experiment set-up
1. Prior to the experiment session
 To be completed at home
 Before deadline
 Software collects data in one-zip file per test
which participants have to submit

25. 25/31
www.janclaes.info
Experiment set-up
1. Prior to the experiment session
 Students will receive their individual results
as incentive to not cheat on these tests
 Late responses or too much errors reveals cheating
 Student is not allowed to start actual experiment if
the test results are not submitted in time

26. 26/31
www.janclaes.info
AT THE EXPERIMENT SESSION

27. 27/31
www.janclaes.info
Experiment set-up
2. At the experiment session
 Modeling task and post-task survey
 In the same, already installed program
(using a different start-up code)
 Software automatically collects data
• Each operation in the tool
( = each activity in the modeling process)
• Constructed process model, number of operations,
modeling time
in one-zip file which they have to submit

28. 28/31
www.janclaes.info
Experiment set-up
2. At the experiment session
 Student does not get credit if not all data (6 zip files)
are submitted
 Technical problems > replacement assignment
(e.g., help me with coding the collected data)

29. 29/31
www.janclaes.info
AFTER THE EXPERIMENT SESSION

30. 30/31
www.janclaes.info
Experiment set-up
3. After the experiment session
 Data will be analyzed and outliners will be defined
 Selection of students will be invited for post-
experiment interview (voluntarily)
 Collect qualitative data
• Explain how they have modeled
• Explain their cognitive profile
• Explain which results we expected and why
• Ask about their opinion: why did they (not) achieve the
predicted results?

31. 31/31
www.janclaes.info
Thanks for you attention!
> Do you have feedback on my research plans? <
> set-up is useful for theory testing? <
> set-up is feasible (enough)? <
> other suggestions? <
Does someone has some time
to test the software? Jan Claes
jan.claes@ugent.be
http://www.janclaes.info
Twitter: @janclaesbelgium

32. 32/31
www.janclaes.info
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
How can the concepts of the theory be measured?

33. 33/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–

34. 34/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
working memory
capacity
Reading span - Operation span - Counting span
Reading span
For many years, my family and friends have been working on the farm. SPOT
Because the stuffy was room, Bob went air for some fresh outside. TRAIL
We were fifty miles out at sea before we lost sight of the land. BAND
Counting span
Count the dark blue circles
Operation span
IS (8 / 4) – 1 = 1 ? BEAR
IS (6 X 2) – 2 = 10 ? BEANS
IS (10 X 2) – 6 = 12 ? DAD

35. 35/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledge
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
input material
representation fit
Fit with modeling task = constant
Fit with modeler through questions
Example questions
I did experience problems using the modeling tool (5-point Likert)
Using the modeling language was not difficult (5-point Likert)
I found it easy to understand the case description (5-point Likert)

36. 36/31
www.janclaes.info
Cognitive Load Theory (CLT)
prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
case complexity
Case complexity = constant
(number of activities and arrows in solution model can be used as proxy for number of concepts and relations)

37. 37/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
prior knowledge
Prior domain, modeling language and modeling method knowledge
through questions
Example questions
I am familiar with the mortgage handling process (5 point-Likert)
How many process models did you ever create?
I am experienced in process modeling (5-point-Likert)

38. 38/31
www.janclaes.info
overall construction effort
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
extraneous
cognitive load
germane
cognitive load
cognitive overload
intrinsic
cognitive load
Through questions
Example questions
I found it hard to complete the task (5-point-Likert)

39. 39/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
– schema construction

40. 40/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
overall construction effort
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
process model quality
Syntactical quality: number of cognitive mistakes
Semantic quality: assessment by experts using checklist (penalty for over- and underfit)

41. 41/31
www.janclaes.info
Cognitive Load Theory (CLT)
case complexity prior knowledge
input material
representation fit
working memory
capacity
extraneous
cognitive load
germane
cognitive load
schema construction
process model quality
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term
cognitive overload
intrinsic
cognitive load
+
+
+
+
+
+
+
+ +
–
–
–
–
–
–
overall construction effort
Time between start of reading case description and last operation in the tool
Number of operations

42. 42/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –

43. 43/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
Through questions
Example questions
When I write a text:
• I usually work on content, structure, wording and format right from the start
• I usually first decide on the content, then the structure, next I write sentences
and finally I work on the formatting

44. 44/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
Through questions
Need for Structure scale
12 questions with 6-point Likert scale

45. 45/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
structuredness
of serialization
– –
– –
Hidden Figures Test

46. 46/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
serialization style fit
– –
– –
degree of
serialization
structuredness
of serialization
Assessment by experts
Flow-oriented – Aspect-oriented – Combination
Happy Path First – Undirected
Uncategorized

47. 47/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
+ + +
structuredness
of serialization
– –
– –
serialization style fit
Theoretical match
Post-experiment interview
Global learner = aspect-oriented, sequential = flow-oriented
High need for structure = no undirected
Field-dependent = flow-oriented

48. 48/31
www.janclaes.info
Structured Process Modeling Theory (SPMT)
A B A determines B
A B The more A, the more B
+ A B The more A, the less B
– A B A translates into B
learning style
degree of
serialization
adopted serialization
style
need for
structure
field-
dependency
– +
+ + +
serialization style fit
structuredness
of serialization
– –
– –
intrinsic cognitive load
for modeling phases
intrinsic cognitive load
for aggregation phases
cognitive overload
intrinsic cognitive load
for strategy building phases
See earlier: through questions