Paper presentation given at the International Conference on Product-Focused Software Process Improvement, 2012

1. Technische Universität München
Who cares about Software Process Modelling?
A First Investigation about the PerceivedValue of Process
Engineering and Process Consumption
Joint work with
Marco Kuhrmann,TUM
Alexander Knapp, University of Augsburg
Daniel Méndez
Technische Universität München
Germany
PROFES 2013
Paphos, Cyprus
13.06.2013
@mendezfe

2. Software Processes
• Blueprint of all relevant artefacts, activities, and roles
➡ Have underlying paradigm:
Context: Software Process Modelling
Activity Orientation
A
Artefact Orientation
B
SW
Process
Analysis
Conceptualisation
Construction
Evaluation
Software Process Modelling
• Systematic design and implementation of a software process
➡ Usually conducted as part of an SPI initiative
A. Activity orientation (e.g., RUP)
B. Artefact orientation (e.g.,V-Modell XT)

3. Software Process Paradigms
Current State of (reported) Evidence
Activity Orientation
Artefact Orientation
• Current view based on mapping study (EASE’ 13)
➡Only few reports on evaluation papers
• Current view based on own experiences & case studies
➡Indicate to benefits regarding quality in artefacts and flexible process

4. Software Process Paradigms
The truth remains...

5. Current studies focus on
• Requirements engineering (having its own particularities) and/or
• Socio-economic contexts with given experiences, expectations and desires about
particular paradigms (based on particular goals)
Why Experimentation?
Taking some steps back
➡ Need for experimentation
• What implications have the paradigms in “nearly context-free” situations:
– No expectations and limited experiences
– Without particular pre-defined improvement goals

6. Experimental Set-Up
Goals and Coarse Setting
Research Objectives
Analyse the perceived value of a chosen paradigm from the perspective of process engineers
and process consumers in context of process life cycle
Working hypothesis
The selection of a paradigm for establishing a process management does not affect its actual
consumption.
Research questions (condensed)
1. How suitable is a paradigm to cover the needs of process engineers?
2. To what extent does a paradigm matter to process consumers?
• Controlled environment / setting
• Pre-defined treatments & assessment criteria
• Randomisation
Still no controlled experiment with
statistical hypothesis testing

7. Process Framework
• Process Frameworks with underlying paradigm-associated meta model and tool support
• Activity orientation: Eclipse Process Framework (Composer)
• Artefact orientation:V-Modell XT (Editor)
Experimental Set-Up
Cases and Subjects
Process
• Workshop organisation process of a German interest group on
“Software Development Processes” / German Computer Society
Subjects
• Two groups covering each both roles
(process engineers and process consumers)
• 8 Students from the course
“Software Engineering Processes”

8. Experimental Set-Up
Data Collection Procedure
Analysis Conceptualisation Construction Evaluation
Workshop 1
Workshop 2
Workshop 3
Audit
Analysis
Concept
Implementation
Concept
Consolidation
Implementation
(Software Process)
Interviews
Process
Engineers
Process
Consumers
Context
• Overview as details of phases
• Covered in lecture
• Conducted as own workshop
Procedure
• Assignment of subjects into two groups /
paradigms
• Consolidation by lecturers
• Cross-examination at the end (audit)

9. Workshops
Goals and Results
Analysis Conceptualisation Construction Evaluation

10. Workshops
Goals and Results
Analysis Conceptualisation Construction Evaluation
Goals
1. Understand process
2. Elicit and sort process elements
input

11. Workshops
Goals and Results
Analysis Conceptualisation Construction Evaluation
Goals
1. First sketch of process elements
2. Clustering and dependencies
Artefacts
Process
structure

12. Workshops
Goals and Results
Analysis Conceptualisation Construction Evaluation
Goals
1.Tool-supported implementation
2. Export process documentation

13. Workshops
Goals and Results
Analysis Conceptualisation Construction Evaluation
Goals
1. Evaluation (engineers)
2. Evaluation/Audit (consumers)
5
5,5
6
6,5
7
Q1-8: Overall
completeness of
artefacts
Q1-9: Completeness
roles
Q1-10:
Completeness
artefacts
Q1-11:
Completeness
relationships
Q1-12:
Completeness
activities
Q1-13:
Completeness
overall process
EPF
V-Modell XT
4,67
5,00
6,00
6,33
7,00
7,00
6,67
5,67
6,33
6,67
6,00
4,20
3,80
3,00
4,00
5,20
6,00
6,20
4,40
3,80
4,75
4,67
Q2-1: HTML export completeness
Q2-2: HTML export accessibility
Q2-3: Overall process presentation
Q2-4: Process verifiability
Q2-5: Implementation completeness
Q2-6: Completeness rolesQ2-7: Completeness artefacts
Q2-8: Completeness relationships
Q2-9: Completeness activities
Q2-10: Implementation
adequateness
Q2-11: Process consistency
EPF VMXT

14. • Artefact-oriented framework supports
– Completeness in artefacts and responsibilities (roles)
– Completeness of relationshps
• Activity-oriented framework supports
– Completeness in activities, but also...
– Overall completeness of artefacts
5
5,5
6
6,5
7
Q1-8: Overall
completeness of
artefacts
Q1-9: Completeness
roles
Q1-10:
Completeness
artefacts
Q1-11:
Completeness
relationships
Q1-12:
Completeness
activities
Q1-13:
Completeness
overall process
EPF
V-Modell XT
Evaluation from Process Engineers

15. 4,67
5,00
6,00
6,33
7,00
7,00
6,67
5,67
6,33
6,67
6,00
4,20
3,80
3,00
4,00
5,20
6,00
6,20
4,40
3,80
4,75
4,67
Q2-1: HTML export completeness
Q2-2: HTML export accessibility
Q2-3: Overall process presentation
Q2-4: Process verifiability
Q2-5: Implementation completeness
Q2-6: Completeness rolesQ2-7: Completeness artefacts
Q2-8: Completeness relationships
Q2-9: Completeness activities
Q2-10: Implementation
adequateness
Q2-11: Process consistency
EPF VMXT
Evaluation from Process Consumers
• Activity-oriented process export overall better rated than artefact-oriented export, e.g.:
– Process consistency
– Completeness relationships
• Not expected: Activity-oriented process export rated as better regarding
– Completeness artefacts
– Completeness roles

16. ?
Summary
• Artefact orientation seems to be perceived of higher value by process engineers
• No similar effects for value perceived by process consumers
– Activity-oriented export rated overall better
– Most surprising: artefact completeness rater better in activity-oriented export
Threats to validity?
• Construct: Completeness of criteria?
• Internal validity: Mistakes during export?
• External: Barely given, but necessary first step!
Our impression: We are still here...
➡Future work:
• Further experimentation
(starter kit available soon!)
• More differentiated view
on paradigms

17. Thank you!