Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

1. Integra(on
tes(ng
in
Enterprises
using
TaaS
Anand
Bagmar
So#ware
Quality
Evangelist
Principal
Consultant,
ThoughtWorks

2. Organiza?on
Objec?ve

3. Business
spread
across
the
globe

4. A
prac?ce
that
makes
Teams
successful?
Test
Automa(on

5. A
prac?ce
that
makes
Teams
unsuccessful?
Test
Automa(on

6. Principles
&
Prac?ces
for
building
a
Test
Automa?on
Framework

7. Principles
&
Prac(ces
of
Automa(on
Framework
Code
Quality
Design
PaBerns
Pairing
Test
Data
Abstrac(on
Layers
Refactor,
Evolve
&
Extend
Configurable
Con(nuous
Integra(on
(CI)
Screenshots,
Video
Recordings
Logging
Tools
&
U(li(es
No
Copy-­‐
Paste

8. Is
Test
Automa?on
treated
as
1st
class
ci?zen
in
your
organiza?on?
• Value
• Quality

9. Quick
survey
• Part
of
a
large
organiza?on?
• #
of
products
in
the
porNolio?
• #
of
projects
across
all
these
products?
• Technology
stack
of
these
products?
• Part
of
merged
/
acquired
companies?
– Same
technology
of
“new”
products?
• Distributed
teams?
• Common
Test
Automa?on
framework?

11. Case
Study

12. Outlook
Outlook
(Windows)
Outlook
(Mac)
Outlook
Web
Access
(OWA)
(Browser)
Outlook
Sync
(Android)
…
…

13. How
will
you
automate
these?

14. Outlook
for
Windows
• White
/
QTP
Outlook
for
Mac
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• Robo?um
/
Java

15. For
a
user
who
has
installed
Outlook
on
– Windows
OS
–
desktop
version,
and,
– Mac
OS
–
desktop
version
– Android
device
–
na?ve
mobile
app
How
will
you
automate
the
test
….
An
email
draQed
in
one
product
is
reflected
in
the
other
products

16. End-­‐2-­‐End
Integra-on
Test
(Test
Framework
for)
Outlook
on
Windows
(Test
Framework
for)
Outlook
Sync
on
Android
Create
&
Save
as
DraE
1
Verify
DraE
4
Verify
DraE
present
2
Modify
DraE
3
Orchestrator

17. Can
you
use
the
same
technology
stack
for
automa(ng
the
tes(ng
of
each
variant
of
Outlook?

18. The
problem

19. Outlook
for
Windows
• White
/
QTP
Outlook
for
Mac
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• Robo?um
/
Java

20. The
solu(on

21. End-­‐2-­‐End
Integra-on
Test
Framework
TaaS
Client
Test
Framework
for
Outlook
on
Windows
TaaS
Server
Test
Framework
for
Outlook
Sync
on
Android
TaaS
Server
Create
&
Save
as
DraE
1
Verify
DraE
4
Verify
DraE
present
2
Modify
DraE
3
Orchestrator
Service
Providers

23. TaaS
Server
• Specify
contract
details
• Implement
contracts
• Return
the
results
• Run
TaaS
Server
(REST
service)
• Serve
TaaS
Client
requests
– As
separate
processes

24. What
is
a
Contract?
Specified
in
a
simple
yml
file

25. Contract
defini?on

26. Contract
defini?on

27. Implement
the
Contract
• Update
tests
to
-­‐
– Handle
input
parameters
• From
Environment
variables
– Execute
the
“test”

28. Implement
the
Contract

29. Parse
Input
parameters

30. Return
the
results
• Output
parameters
– As
console
output
between
special
markers
• Console
logs
• Excep?ons
(if
any)
• All
return
values
are
in
“json”
format

31. Return
Output
parameter
values

32. Run
TaaS
Server
• Dedicated
machine
• Provide
IP
address
/
host
name
+
port
to
poten?al
clients

33. TaaS
Client
• Collect
the
input
parameters
• Invoke
the
contract
• Consume
the
result
• Con?nue
with
test
orchestra?on

34. Invoke
TaaS
Contract

35. Summary
of
Features
• Developed
in
Ruby
using
Sinatra
• Contract
– Decoupling
of
technology
barriers
– Timeout
• Passing
of
input
parameters
as
Environment
variables
• Result
as
json
– Output
parameters
– Console
logs
and
errors
– Excep?ons

36. Why
is
this
a
good
idea?
• Automate
the
last-­‐mile
• No
code
duplica?on
• Implementa?on
of
contract
lies
with
the
framework
tes?ng
that
product
– Evolves
with
product
changes
• Decoupling
of
technologies

37. Why
is
this
a
good
idea?
• Helps
in
Manual
Tes?ng
(setup
of
data)
• Anyone
can
use
it
• Each
product
is
tested
in
the
“best”
possible
way
• Its
–
providing
Test
as
a
Service!!!

38. What
this
is
Not
• Load
tes?ng
tool
• A
“tool”
for
integra?ng
different
products
– Lack
of
security
– Probably
not
as
robust

39. What
TaaS
did
for
me?
• Be
crea?ve
• Find
Innova?ve
solu?on
to
the
problem
• Another
open
source
contribu?on

40. Does
it
really
work?

41. How
can
you
use
TaaS?
• Open
source
(apache2
license)
• Available
on
github!
• Available
as
taas.gem
on
rubygems.org
(~500
downloads)

42. • Error
codes
for
the
service
• Service
Discovery
• Security
• Create
a
Java
/
.Net
version
of
TaaS
Server
How
can
you
help
TaaS?

43. hkp://essenceo#es?ng.blogspot.com
hkps://github.com/anandbagmar/taas
anandbagmar/
Anand.Bagmar@thoughtworks.com
Abagmar@gmail.com
@BagmarAnand