Felipe Besson Masters defense

1. Rehearsal
A framework for automated testing of
web service choreographies
Felipe M. Besson
besson@ime.usp.br
Masters Defense
Advisor: Fabio Kon
September 14th
, 2012
IME-USP

2. 2
Web services
●
Interoperable communication
●
Machine readable interface
RSS google
calendar google
places
●
In SOA context:
●
Soap/WSDL web services
●
A WS can be composed in a more complex one

3. 3
Orchestration
●
Centralized
●
Process
●
BPEL and SCA
A
●
Difficult to manage
●
Less scalable
B
C

4. 4
Choreography
●
Decentralized
●
Distributed
coordination
●
WS-CDL and BPMN 2
A
B
C

5. 5
Choreography
●
Roles: interface (WSDL) and behavior
●
Distributed coordinators playing roles (Peltz, 2003)
Store Bank
Shipper

6. 6
Problems of choreographies
●
Development
●
Complex
●
Focus on modeling
●
No widely-adopted standards
●
Testing
●
Parallel and many messages
●
Binding problems
●
Third-party governance rules

7. 7
Consequences
●
A lack of
●
development methodologies and processes
●
tools for automated testing of WS compositions
●
tools for supporting development (e.g., debugging)
●
Consequences
●
Ad hoc choreographies
– Low quality
●
Low adoption

8. 8
Our Goals
A testing framework to support Test-Driven
Development (TDD) of choreographies
services and
roles in isolation
messages
exchanged entire
choreography

9. 9
Test-Driven Development (TDD)
●
A design technique that drives the development
process through testing [1]
RefactorRefactor CodeCode
TestTest
1. Write an AUTOMATED
test for next functionality
you want to add;
2. Write the functional code
until the test passes;
3. Refactor the new and old
parts of the code.
[1] (Fowler, 2001; Beck, 2003)

10. 10
Motivation
●
Most existing efforts [2] for testing choreographies
●
Focus on validation and simulation of models
●
None of them focus on TDD
●
Aggregate more importance to development
activities
●
TDD can facilitate the choreography development
and leverage its adoption
[2] (Bucchiarone, 2007; Canfora, 2009; Palacios, 2011)

11. 11
Agenda
●
Rehearsal
●
TDD methodology proposal
●
Exploratory study
●
Conclusions

12. 12
Rehearsal

13. 13
Choreography testing
●
Surveys (Bucchiarone, 2007; Canfora, 2009)
Unit
(Peltz, 2003)
Integration Acceptance

14. 14
Unit and Acceptance testing
●
WSClient: a dynamic generator of web service
clients
●
Related work
●
SoapUI (SmathBear, 2012)
●
SOCT (Bartolini, 2009)
●
BISTWS (Medeiros, 2010)
●
Drawbacks:
●
Tests written in XML
●
The service contract must be available

15. 15
WSClient
●
Testing web services without having the
contract
●
Service operations are validated at runtime

16. 16
WSClient

17. 17
WSClient
Item explorer
Creating
Item objects
automatically

18. 18
Integration testing
●
Message Interceptor
●
Related work
●
Pi4SOA (Pi4, 2012)
●
CDL-Checker (Wang, 2010)
– Messages are validated by simulation
●
BPEL Unit (Mayer, 2006)
– Test cases are written in XML
– Coupled to BPEL engines

19. 19
Message Interceptor
Intercepting, storing and validating messages
exchanged among services and roles

20. 20
Message Interceptor
A B
D
C E F

21. 21
Integration testing
●
How to deal with thirdy-party constraints ?
●
Governance rules
●
Absence of testing environment
●
WSMock: service mocking
●
Related Works
●
SoapUI (SmartBear, 2012)
– Difficult to manage and customize
– Basis for the WSMock implementation

22. 22
WSMock
Simulating regular and faulty scenarios

23. 23
WSMock

24. 24
Abstraction of Choreography
Abstracting the choreography elements into
objects
●
Test cases can be written easily and clearly
●
Help to understand the choreography execution
●
Relate the test cases to the choreography models
(diagrams)
BPMN 2
diagrams Rehearsal
Java Objects
Roles
Services
Messages
tests

25. 25
Abstraction of Choreography
Future: bookTrip = new Choreography(“book.bpmn2”);
Service airline1 = bookTrip.getServicesByRole(“airline”).get(0);
WSClient ws = new WSClient (airline1.getWSDL());
Item flight = ws.request(“searchFlight”, “São Paulo”, “Paris”, … );
assert(“O-815”, flight.getContent(“id”));
Actual: bookTrip = new Choreography(“descriptor.yml”);

26. 26
TDD Methodology
proposal

27. 27
Creation or adaptation of atomic web services
– Choreography developed from scratch
●
Contract is defined by the tests
– Other cases
●
Tests guide the development
●
Tests are an executable documentation
– WSClient!
Phase 01

28. 28
Example of executable specification
Phase 01

29. 29
Phase 2: services to compose the roles
Phase 3: roles to compose the choreography
– Validate the messages exchanged
inside/outside the executable processes
●
Mocking real dependencies
●
Using the Message Interceptor for
inspecting and validating the messages
– Detect binding problems early
Phases 2 and 3

30. 30
Acceptance and Scalability Testing
– Validate all choreography features from the
user point of view
●
Trying to use an environment closer to the
production
– Assess the choreography scalability
●
large workload and different choreography
configurations
Phase 4

31. 31
Exploratory Study

32. 32
●
Subjects: 21 CS students of IME-USP
●
2 phases
●
Varying the experience in web service
development and TDD
●
Systematic protocol
Design
( 4 tasks )

33. 33
RQ1: Does the Rehearsal features aid in the
application of the proposed methodology steps ?
RQ2: Does the proposed methodology provide
adequate guidelines for developing a
choreography ?
Phase 1
Assessing the adequacy of using Rehearsal
following the TDD methodology proposal

34. 34
Experience # 8 students
participants
participants
participants
participants
participants
Software development (years) TDD (years) Java (years)
WS development (years) WS compositions (years)
1-3 3-5 more than 5 0-2 more than 3 1-3 3-5 more than 53-5
0-10more than 21-20-1

35. 35
RQ1: How easy is to use the Rehearsal features to
apply the methodology steps ?
RQ2: How easy is to follow the methodology steps
to develop a choreography ?
Phase 2
Assessing how easy is to use and learn Rehearsal
and the TDD methodology

36. 36
Experience # 11 students
participants
participants
participantsparticipants
Software development (years) TDD (years)
WS development (years) Java (years)
1-3 3-5 more than 5
0 0-1 1-2
1-2 more than 20-10 1-3 more than 5

37. 37
Example of Question
The questions of each statement was
provided following the scale:
5. Strongly Agree
4. Partially Agree
3. Indifferent
2. Partially Disagree
1. Strongly Disagree

38. 38
Qualitative Results
Percentile(25,75) of QtiPlot

39. 39
Rehearsal – WSClient
●
QR1: It was easy to learn how to use the WSClient feature
●
QM3 (p1): The use of the WSClient feature is useful in Task 01
●
QM3 (p2): It was easy to use the WSClient feature in Task 01
●
QM13 (p1): The use of the WSClient feature is useful in Task 04
●
QM13 (p2): It was easy to use the WSClient feature is useful in Task 04
questions - phase 1 questions - phase 2

40. 40
Rehearsal – WSMock
●
QR2: It was easy to learn how to use the WSMock feature
●
QM6 (p1): The use of the WSMock feature is useful in Task 02
●
QM6 (p2): It was easy to use the WSMock feature in Task 02
questions - phase 1 questions - phase 2

41. 41
Rehearsal – Abstract of choreography
●
QR4: It was easy to learn how to use the Abstraction Choreography
●
QR5: The Abstraction Choreography feature helped me to use other
Rehearsal features
●
QR6: The Abstraction Choreography feature helped me to write the
test cases
questions - phase 1 questions - phase 2

42. 42
Methodology – Acceptance
●
QM14: I think the use of the methodology and Rehearsal would be
useful for projects I have participated
●
QM15: I would use the methodology and Rehearsal in future
projects I may participate.
questions - phase 1

43. 43
Contributions
●
An Open Source Testing Framework
●
Examples of web service choreographies
●
Contributions to the SoapUI community
●
Exploratory study package
●
Protocol
●
Training material
●
Results
●
Produced code

44. 44
Contributions
Felipe M. Besson, Pedro M. B. Leal, Fabio Kon, Alfredo Goldman and
Dejan Milojicic. Towards automated testing of web service
choreographies. 6th
International Workshop on Automation of
Software Test (AST'11) on ICSE. Waikiki, Honululu, HI, USA, 2011.
Felipe M. Besson, Pedro M. B. Leal, Fabio Kon, Alfredo Goldman and
Dejan Milojicic. Supporting Test-Driven Development of Web
Service Choreographies. 5th
Open Cirrus Summit, Moscow, Russia,
2011.
Felipe M. Besson, Paulo Moura, Fabio Kon, Dejan Milojicic.
Rehearsal: A framework for automated testing of web service
choreographies. 3th
Brazilian Conference in Software Theory and
Pratice, Tool Session, Natal, Brasil, 2012. (to be presented)

45. 45
Future Work
Improve Rehearsal functionalities
●
Abstraction of choreography
●
Dynamic reconfiguration
●
Generate test cases automatically
●
Testing coverage
Improve TDD methodology
●
Integrate to choreography modeling
Software Engineering Experiments
●
Compare with ad hoc development
methodologies

46. 46
Conclusions
Our goals have been reached =]
●
Good results in the exploratory study
– For experienced and non experienced
developers
●
Relevant contribution to the topic (academy
and industry)
Rehearsal and TDD methodology can aggregate
more importance to the developer role in the
life cycle of SOC applications

47. 47
Thanks a lot !!!
●
My parents
●
My girlfriend Jaq S2
●
My advisor Fabio
●
Friends of IME-USP
●
Friends of Elo7

48. 48
Thanks to
●
HP Brasil under the Baile Project
●
European Community's Seventh Framework Program
FP7/2007-2013 under grant agreement number 257178
(project CHOReOS - Large Scale Choreographies for the
Future Internet)
Acknowledgements
Specially to
Dejan Milojicic

49. 49
Chris Peltz. Web Services Orchestration and Choreography. Computer,
36:4652, October 2003.
Martin Fowler. Test-Driven Development. Available on:
http://www.martinfowler.com/bliki/TestDrivenDevelopment.html, 2011.
Kent Beck. Test-driven development: by example. Addison-Wesley, Boston,
2003.
Antonio Bucchiarone, Hernán Melgratti, and Francesco Severoni. Testing
Service Composition. In 8th Argentine Symposium on Software Engineering
(ASSE'07), Mar del Plata, Argentina, 2007.
Gerardo Canfora and Massimiliano Di Penta. Service-Oriented Architectures
Testing: A Survey. In Software Engineering, volume 5413 of Lecture Notes
in Computer Science, 2009.
Marcos Palacios, José García-Fanjul, and Javier Tuya. Testing in Service
Oriented Architectures with dynamic binding: A mapping study.
Information and Software Technology, pages 171189, March 2011.
References

50. 50
SmartBear. SoapUI, Web Services Functional Testing Tool. Available on:
<http://http://www.soapui.org/>, 2010.
Cesare Bartolini, Antonia Bertolino, Sebastian Elbaum e Eda Marchetti.
Whitening SOA testing. In Proceedings of the the 7th joint meeting of the
European software engineering conference and the ACM SIGSOFT Symposium
on The foundations of software engineering, ESEC/FSE '09, New York, NY, USA,
2009.
Marcelo Medeiros Eler, Marcio Eduardo Delamaro, Jose Carlos Maldonado e
Paulo Cesar Masiero. Built-In Structural Testing of Web Services. Em
Proceedings of the Brazilian Symposium on Software Engineering, SBES '10.
IEEE Computer Society, 2010
Pi4 Technologies Foundation. Pi calculus for SOA. Available on:
http://sourceforge.net/projects/pi4soa/, 2010.
San-Yih Hwang, Wen-Fan Hsieh e Chien-Hsiang Lee. Verifying web services in a
choreography environment. Em SOCA. IEEE, 2011.
Philip Mayer e Daniel Lübke. Towards a BPEL unit testing framework. Em
Proceedings of the 2006 workshop on Testing, analysis, and verification of web
services and applications, TAV-WEB '06, New York, NY, USA, 2006
References