EuroSTAR Software Testing Conference 2008 presentation on Test Environment, The Future Achilles’ Heel by Mickiel Vroon. See more at conferences.eurostarsoftwaretesting.com/past-presentations/

1. Test environment
Achilles heel of the future?
Michiel Vroon

2. Test environment

3. Sounds familiar?
• I thought we solved this defect before?
• This part worked fine some moment ago!
• Who knows the correct configuration?
• What do you mean, first production??!!
• This looks different from production!
• Do you know a user-id we can use?
• What is in the current release?
• I cannot reproduce the defect!

4. Some causes
• No dedicated competence
• Difficult to define requirements for test
• There are no processes, plan or strategy
• Design is not clear
• Test data is not maintained
• Too many people involved
• Responsibilities are vague
• Manageable versus flexible
• …

5. Some solutions
• Define single point of contact
• Set up processes
• Arrange test data
• Work with environment types
• Use master/copy
• Install shared environments
• Have an authorization strategy

6. Define single point of contact
• Test environment coordinator
– Responsible for setting up and maintaining environment
– Single gateway to all parties involved
– Combination of technical and testing skills
– Control function
– Communicator and project leader
– In bigger organizations:
dedicated department of
test environments

7. Set up processes - 1
• Define what to deliver when and by whom
• Connection with other processes: design, build,
test and implementation
• Simple process:
1. Specifying the infrastructure
2. Realising the infrastructure
3. Specifying the infrastructure intake
4. Intake of the infrastructure
5. Maintaining the infrastructure
6. Preserving the infrastructure
2 3
5
Start
4
1
6
End

8. Set up processes - 2
Maintaining the environment
– Configuration management
– Change management
– Release management
Wish Use
Dev Test Acc Prod
Configuration management
Change
Release

9. Arrange test data
• Define test data
– Production data (real or scrambled)
– Test data (regular system functions or separate)
• Naming test data
– Real names
– Functional names
– Test related names
• Maintain test data
– Cumulative
– Periodic restore
– Several versions

10. Work with environment types
• DTAP –Model, 4 types of environments
– Development
– Test
– Acceptance
– Production
• Not a technical solution!
• Each type has it’s own characteristics on:
– Maintenance and ownership
– Authorization
– Flexibility

11. Example
Local
development
ST
environment
UAT
environment
Development Test Acceptance Production
Environment in
project
Environment type
acc. to DTAP
Central
development
PAT
environment
Production
environment
Shadow
environment

12. Use master/copy
• Principle for setting up environments
Master
Copy
Copy
Copy
• What is the master?
• What is in the master?
• Who owns the master?

13. Install shared environments
• An end-to-end environment
• Consistent end-to-end test data
• Shared use by different projects
• Well defined user processes
• Always up and running
• Continuous maintenance

14. Have an authorization strategy
• How to deal with authorization in the
environment?
• Test users or real life users?
• Impersonal users (roles like “guest”) or
personal users?

15. The case!
ITO Project
Integrale Test Omgeving
(Integral Test Environment)

16. The situation
• Dutch bank
• Department Environments
– Responsible for all test environments

17. Single point of contact
Department environments
Coordinator
environments
Maintenance
&
Exploitation
Outsource
parties
Projects
External
partners
Delivery
unit
Unix
Local
Central
Test data

18. Processes

19. Current situation
• Availability environments
– Demand > supply
– Temporary solutions
• Maintainability environments
– Versions
• Test data environments
– End-to-end consistency
• Complexity
– Replacement old client systems (OLI > SIEBEL)
– Connections

20. The environments
OLI (25)
X (25)
Y (20)
Z (15)
A (10)
B (5)
Siebel (2)

21. Replacement old client systems
OLI (25)
X (25)
Y (20)
Z (15)
A (10)
B (5)
Siebel (25)

22. end-to-end shared environments
OLI (5)
X (5)
Y (5)
Z (5)
A (5)
B (5)
Siebel (5) Connected systems
Consistent test data
Maintained
Shared use
Rules

23. Goals
• Reduction of systems
• Transparency
– You know what’s in the environment (data, software,
hardware, connections etc)
– It’s consistent
– All users are known
• Shorten setup time
– It’s already there
– Only adjustments of connections

24. Results

25. Quaboo
Ludwigstraat 39
4701 NE Roosendaal
The Netherlands
E. michiel@quaboo.eu
M. +31(0)6 21 489 775
I. www.quaboo.eu