UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
Deploy and Destroy Complete Test Environments
1. Deploy and Destroy Complete
Test Environments
Service Virtualization, Containers and Cloud
Bas Dijkstra
bas@ontestautomation.com
www.ontestautomation.com
@_basdijkstra
4. Test environment impediments
System under test
Mainframe
SaaS
dependency
Backend
system
Mobile app
No suitable test data Limited access
Under development Access fees
5. The T-shaped tester: What about
the T-shaped test environment?
Under
control
100%
availability
Scalable
Realistic
performance
Suitable
test data
6. Service virtualization
System under test
Virtualized
mainframe
Virtualized
SaaS
dependency
Virtualized
backend
system
Virtualized
mobile app
Unrestricted access
Unrestricted access
Unrestricted access
Unrestricted access
17. Example: Parasoft Virtualize
_Support for many message types and protocols
_Commercially licensed
_Environment Manager
_http://www.parasoft.com/virtualize
18. Virtualize + VSTS + Azure
commit
Parasoft
Environment
Manager
Simulated test environment
Application under test
AUT
Parasoft
SV
Azure cloud VM
Host OS
Server
On-demand test
environment in
the cloud
Everybody is doing some sort of CD, or at least is looking into it
CD requires the ability to test continuously, every deployment should be thoroughly tested before being put into a production environment
Test automation plays a big role in this
The ability to test continuously is often hard to achieve because of test environment impediments
The T-shaped tester is a sought after profile at the moment, very good in one thing (testing, automation) and skilled in myriad other things (analysis, development, …)
Continuous Testing requires something similar: a T-shaped test environment. First and foremost it should be under total control of the team using it, but next to that it should have other characteristics as well..
One approach to enable Continuous Testing is through implementation of service virtualization, an approach that removes bottlenecks in test environments by virtualizing dependencies that are critical to the testing process yet hard-to-access
Characteristics of service virtualization
R&P: draw parallel to test automation: nice to get a first demo up and running (or analyse traffic where no specifications are known), but to achieve long term success it’s best not to rely on R&P too much. Instead, create virtual assets using request-response specifications. R&P also only simulated those situations that CAN be recorded.
How should you go about SV implementation? Start small, both in terms of number of dependencies simulated and behavior replicated. Model just enough to enable execution of desired test cases (all else is filler, remember that SV is an enabler for testing, not a goal in itself). Learn and expand.
So how does the testing process ultimately benefit from SV implementation?
So, we have seen that implementing SV can lead to great benefits for the testing process, but to make SV even more flexible and better integrated into the CD process, wouldn’t it be nice if we could treat our test environment in the same manner as our application under test?
By containerizing and distributing SV engines and their virtual assets, we can do…
So, what does an approach like this look like? Talk them through the picture. Build server also runs tests (unit tests), we’re talking about integration and end-to-end tests here.
Guarantees that an application will always run in exactly the same way, regardless of the environment it’s running in. This is what you want from your test environment as well.
Hoverfly, mostly focuses on record and playback functions by acting as a proxy, recorded traffic can be extended and customized programmatically
Also supports latency simulation for more realistic performance testing
Tell a little about Virtualize as a tool
Both EM and Virtualize engine can be deployed directly from Azure, EM can then provision simulated test environments on the fly
Talk them through the picture, focus also on repeatabiity
So we’ve decided that SV might be the way to go, but what decisions do we need to make? And of course money does play a role…
Where do I host the SV server / the virtual test environments?
Who is made responsible for developing and maintaining the virtual assets?
Is virtual asset development done in house (requires SV expertise, which, although it isn’t rocket science, might be hard to come by) or is it outsourced to a third party?
Much like regular software development
What could an example maturity journey look like?
From single server, developed and maintained in one or more teams (small scale) to cloud hosted (flexible, scalable) developed by a center of excellence (less overhead, less fragmentation, one way of working)