Slides used in Agile2013 conference on the topic - Integration Testing in Enterprises using TaaS - via Case Study

1. Integra(on
Tes(ng
in
Enterprises
using
TaaS
Anand
Bagmar
So#ware
Quality
Evangelist
Principal
Consultant,
@BagmarAnand

2. Reality
–
OrganizaBons
are
spread
across
the
World

3. OrganizaBon
ObjecBve
Time
Quality
Money

4. What
is
a
pracBce
that
makes
Teams
successful?
Test
Automa(on

5. What
is
a
pracBce
that
makes
Teams
unsuccessful?
Test
Automa(on

6. Is
Test
AutomaBon
treated
as
1st
class
ciBzen
in
your
organizaBon?
• Value
• Quality
Where
is
Test
AutomaBon
placed
in
the
Food
Chain
of
SDLC?

7. Principles
&
PracBces
for
Building
a
Test
AutomaBon
Framework

8. Principles
for
building
an
Automa(on
Framework
Code
Quality
Design
PaBerns
Abstrac(on
Layers
Refactor
Pairing
Evolve
Extend

9. Prac(ces
for
building
an
Automa(on
Framework
Configurable
Con(nuous
Integra(on
(CI)
Screenshots,
Video
Recording
Logging
Tools
&
U(li(es
Test
Data
No
Copy-­‐Paste

11. Case
Study

12. Outlook
Outlook
(Windows)
Outlook
(Mac)
Outlook
Web
Access
(OWA)
(Browser)
Outlook
Sync
(Android)
…

13. How
will
you
Automate
the
TesBng
for
Outlook?

14. Outlook
for
Windows
• White
/
QTP
Outlook
for
Mac
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• RoboBum
/
Java

15. For
a
user
who
has
installed
and
configured
Outlook
on
– Windows
OS
–
desktop
version,
and,
– Android
device
–
naBve
mobile
app
– Mac
OS
–
desktop
version
– …
An
email
draQed
in
one
product
is
reflected
in
the
other
products
How
will
you
Automate
the
IntegraBon
Test
….

16. End-­‐2-­‐End
Integra-on
Tests
(Test
Framework
for)
Outlook
on
Windows
(Test
Framework
for)
Outlook
Sync
on
Android
Create
&
Save
as
DraE
1
Verify
DraE
4
Verify
DraE
present
2
Modify
DraE
3
Orchestrator

17. Can
you
use
the
same
technology
stack
for
automa(ng
the
tes(ng
of
each
variant
of
Outlook?
The
Problem

18. Outlook
for
Windows
• White
/
QTP
Outlook
for
Mac
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• RoboBum
/
Java

19. My
Original
Problem

20. The
Solu(on

21. What
is
TaaS?
“TaaS”
is
a
product
that
allows
you
to
validate
the
integra-on
across
a
variety
of
products
via
Test
Automa-on
–
the
“correct”
way

22. What
is
TaaS?
Agnos-c
Pla^orm
/
OS
ImplementaBon
Language

23. What
is
TaaS?
Contract
Decoupling
of
technology
barriers
Timeout

24. What
is
TaaS?
Simple
Parameters
Input
&
Output

25. What
is
TaaS?
Developed
in
Ruby
using
Sinatra

26. TaaS
is
Not
a
…
“Tool”
for
integraBon
different
products
-­‐ Lack
of
Security
-­‐ Probably
not
as
robust

27. TaaS
is
Not
a
…
Load
TesBng
Tool

28. How
does
TaaS
work?

29. End-­‐2-­‐End
Integra-on
Tests
Framework
TaaS
Client
Test
Framework
for
Outlook
on
Windows
TaaS
Server
Test
Framework
for
Outlook
Sync
on
Android
TaaS
Server
Create
&
Save
as
DraE
1
Verify
DraE
4
Verify
DraE
present
2
Modify
DraE
3
Orchestrator
Service
Providers

30. TaaS
Server
ImplementaBon

31. What
is
a
Contract?
Specified
in
a
simple
yml
file

32. Contract
DefiniBon

33. Contract
DefiniBon

34. Implement
the
Contract
• Update
tests
to
-­‐
– Handle
input
parameters
• From
Environment
variables
– Execute
the
“test”

35. Implement
the
Contract

36. Parse
Input
Parameters

37. Return
the
Results
• Output
parameters
– As
console
output
between
special
markers
• Console
logs
• ExcepBons
(if
any)
• All
return
values
are
in
“json”
format

38. Collect
Output
Parameter
Values

39. Return
Output
Parameter
Values

40. Run
TaaS
Server
• Dedicated
machine
• Provide
IP
address
/
host
name
+
port
to
potenBal
clients

41. TaaS
Client
ImplementaBon

42. Setup
TaaS
Client

43. Invoke
TaaS
Contract

44. TaaS
Technical
Summary

45. TaaS
Server
• Specify
contract
details
• Implement
contracts
• Return
the
results
• Run
TaaS
Server
(REST
service)
• Serve
TaaS
Client
requests
– As
separate
processes

46. TaaS
Client
• Collect
the
input
parameters
• Invoke
the
contract
• Consume
the
result
• ConBnue
with
test
orchestraBon

47. Summary
of
Features
• Contract
– Decoupling
of
technology
barriers
– Timeout
• Passing
of
input
parameters
as
Environment
variables
• Result
as
json
– Output
parameters
– Console
logs
and
errors
– ExcepBons
• Developed
in
Ruby
using
Sinatra

48. Why
is
this
a
Good
Idea?
• Automate
the
last-­‐mile
• No
code
duplicaBon
• ImplementaBon
of
contract
lies
with
the
framework
tesBng
that
product
– Evolves
with
product
changes
• Decoupling
of
technologies

49. Why
is
this
a
Good
Idea?
• Helps
in
Manual
TesBng
(setup
of
data)
• Anyone
can
use
it
• Each
product
is
tested
in
the
“best”
possible
way
• Its
–
providing
Test
as
a
Service!!!

50. How
can
you
use
TaaS?
• Open
source
(apache2
license)
• Available
on
github!
• Available
on
rubygems.org
!

51. Steps
to
Follow
• NegoBate
contract
details
• Setup
&
configure
TaaS
Server
– Implement
contracts
– Return
the
results
– Run
TaaS
Server
(REST
service)
• Configure
TaaS
Client

52. Does
it
really
work?
>550
downloads

53. Before
you
use
TaaS
…
IdenBfy
–
• Is
TaaS
the
right
thing
for
you?

54. • Error
codes
for
the
service
• Service
Discovery
• Security
• Create
a
Java
/
.Net
version
of
TaaS
Server
• Asynchronous
requests
How
can
You
help
TaaS?

55. What
TaaS
did
for
me?
• Be
creaBve
• Find
InnovaBve
soluBon
to
the
problem
• Another
open
source
contribuBon

56. abagmar@gmail.com
@BagmarAnand
hip://about.me/anand.bagmar
Thank
you