SlideShare ist ein Scribd-Unternehmen logo
1 von 59
Downloaden Sie, um offline zu lesen
Continuous
Delivery
Continuous Testing
Tomas Riha
Architect @ VGT/WirelessCar
Passionate about creativity, change and improvement
Horrible at following instructions and performing repetitive tasks
MAJOR Project Liability
mail: triha74@gmail.com
twitter: @TomasRihaSE
blog: continuous-delivery-and-more.blogspot.com
Agenda
Why do we want to do Continuous Delivery
Intro to Continuous Delivery
Principles of Continuous Delivery
Look at a Pipe
Test Automation for Continuous Delivery
Test Driven Development and Continuous Delivery
Exploratory Testing and Continuous Delivery
Some words on Tooling
Areas not covered
Why do we want to do Continuous Delivery
Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.
Why do we want to do Continuous Delivery
Decreased lead time not only improves our time to market
but also our ability to learn and improve through a faster
feedback loop
Why do we want to do Continuous Delivery
As soon as we have added value to the application it needs
to be in the hands of the customer or else we have just
added cost.
Why do we want to do Continuous Delivery
Reduced risk through delivery of small changes.
Are we really delivering as soon as possible?
Pre
Planning
Dev Sys Test Reg Test
Pre
Planning
Dev Sys Test Reg Test
Scrummerfall happens because its hard for developers to have
something deployable for the testers to test.
System testing on something that has not been regression tested is
fundamentally flawed.
Cost of regression test tends to bloat sprint content.
Sprint 2-4 weeks
Continuous Delivery
Build Release Deploy Test Prod
Your application always builds, passes its unit tests,
deploys and passes system tests.
Continuous Integration/Delivery/Deploy
Continuous Integration
Your application always builds and passes its unit tests.
Continuous Delivery
Your application always builds, passes its unit tests, deploys and passes its
system tests making it always ready to deploy to production through an
automated process
Continuous Deployment
Your application automatically deploys to production through a fully
automated process
Continuously as in all the time?
Yes!
End of sprint isn't continuously.
Weekly isn't continuously.
Nightly isn't continuously.
Every code commit is continuously!
What Tests? Unit? Component? System?
Yes!
Unit
Component
System
Integration
Rollback
Load
Failover
Principles of Continuous Delivery
1. The process MUST be repeatable, reliable and
independent of key personnel
2. Automate everything
Principles of Continuous Delivery
3. If something is difficult or painful, do it more often
Principles of Continuous Delivery
4. Keep everything in source control AND release it
Principles of Continuous Delivery
5. Done means released
Principles of Continuous Delivery
6. Build Quality In
Principles of Continuous Delivery
7. Fail Fast
Principles of Continuous Delivery
8. Everybody has responsibility for the release process
Principles of Continuous Delivery
9. Improve continuously
Principles of Continuous Delivery
The pipe - Build once!
Build Release
Build and Unit test then release it!
The pipe - Release everything!
Build Release
Build Release
Build Release
Build Release
Code
DB Scripts
Server Config
Deploy script
Feedback Feedback
The pipe -Bundle released artifacts
Build Release
Build Release
Build Release
Build Release
Assembly
Feedback
The pipe - Use Same Deploy Mechanism
Function TestAssembly Deploy
DB
Server
Function Test
Function Test
Pipe Status
Feedback
Feedback Feedback
The pipe - Use Same Deploy Mechanism
Deploy
DB
Server
Pipe Status
Server
DB
Server Server
DB
Server Server
Deploy
Pre Prod Prod
UAT
FeedbackFeedback Feedback
Continuous Delivery
Build Release Assemble Deploy
Deploy
PreProd/Prod
Test
Summary
We build once
Release everything
Automate everything
Fast feedback
Automate all tests except UAT
Obvious benefits
Continuous Regression Testing gives instant feedback.
Continuously deploying to test servers tests deploy mechanism several
hundred times per release.
Always ready to push new release into UAT
We can get customer feedback as soon as value has been added
Automate all tests!
Wait a minute lets look at what we have been doing in the past.
Testing in the past
How its been done
SYSTEM
GUI
Testing in the past
What we have done
100s of manual tests that..
... verify our requirements
... ensures bad input doesn't break system
... data is stored in the right place
... find bugs
Testing in the past
We have automated it!
We have tried to automate the 100s of manual tests that..
... verify our requirements
... ensures bad input doesn't break system
... data is stored in the right place
... find bugs
We do it through instrumenting the GUI and verifying
Database, GUI and System resources.
With very bad results....
Robust and Fast Test Automation
Tests that execute 100s of times per day have to be...
... executable without any human interaction
... robust and give the same response every single time
... robust and have to survive refactoring
... fast and parallelizable as adding 10 tests should not slow
down the feedback
We need to change the way we look at Test
On System Level we do verification of Use Case
Acceptance Criterias. Requirement Verification.
On Component Level we do testing of Component
Functionality. Testing.
We need to change the way we look at
Architecture
Remember “Build quality in”.
We need to improve testability of our applications through...
...isolation of functionality into independent services with
life cycles of their own.
...building interfaces for defined test points.
Building testable systems
Internal
Service
GUI
Internal
Service
Internal
Service
Service API
Component Tests
Internal
Service
Internal
Service
Internal
Service
Each Service Tested as Black Box in isolation
and with high detail level
GUI is a component
GUI
Service API Mock Impl
GUI Test are much faster and robust if they test the GUI in isolation.
High detail level.
Requirement Verification
Internal
Service
GUI
Internal
Service
Internal
Service
Service API
Verifying Use Case
acceptance criterias.
Low level of detail
Continuous Delivery
Build Release Deploy Test
Build Release Deploy Test
Build Release Deploy Test
Build Release Deploy Test
Build Release Deploy Test
Assemble Release Deploy Test
Component Pipe
100s of tests run in parallel testing
System Pipe
10s Use Case Requirements Verified
Test Driven Development
Pre
Planning
Dev Sys Test Reg Test
Pre
Planning
Dev Sys Test Reg Test
Scrummerfall happens because its hard for developers to have
something deployable for the testers to test.
System testing on something that has not been regression tested is
fundamentally flawed.
Cost of regression test tends to bloat sprint content.
Sprint 2-4 weeks
Test Driven Development
Pre
Planning
Dev
Test
Automation
Reg Test
Design Requirement Verification and Tests on multiple levels with the feature.
Develop test automation in parallel with feature development.
Cross functional pairing to improve quality of Tests, Test Architecture and Feature
Testability
Pre
Planning
Reg Test Reg Test
Verification
Verification
Test Driven Development
Who should automate the tests?
Test Design - Done by Testers
Test Automation - Done by ????
Coding - Done by Developers
Test Driven Development
Tests as Code
Test Automation IS Code and needs to be treated as first
class application.
Test Architecture is super important and needs to be
worked with in parallel with Application Testability
Architecture.
Test Driven Development
Tests as Code
Internal
Service
DSL
Glue Code
SpecificationText or Code based Specification
Implementation of Specification
Domain Specific Language packaging
service testability
Test Driven Development
Internal
Service
DSL
Glue Code
Specification
Give a valid user name and password
when login is called
then user is authorized
boolean authorized = loginService.authorizeUser(“triha”,”cdrockz”);
assert(authorized, true);
public boolean authorizeUser(String userName, String password){
//Implementation of REST call to internal service
return true;
}
Test Driven Development
Tests as Code
Specification & Glue Code IS the Test
DSL is testability
Test Automation is Code
Test Driven Development
Pre
Planning
Dev
Test
Automation
Reg Test
Specifications of Requirement Verification written during Pre-Planning.
Specification of Component Tests written during Pre-Planning.
Implementation of DSL and Glue Code in parallel with feature development.
Implementation of Component Tests in parallel with feature development.
Pre
Planning
Reg Test Reg Test
Verification
Verification
Test Driven Development
Tomorrow Agile Team Member
Developers with Testing skills
Testers with Development skills
WILL BE SUPER HOT AND EARN SUPER NICE $$$$
Will be the only ones who get assignments
Exploratory Testing
Automate Everything!
Yes!
Everything that needs to be done in order to release a
system into production.
Yet we still need Exploratory testing.
Exploratory Testing
Has been used to cover up manual regression testing
or worse
A way to get away with non reproducible manual regression
testing
Exploratory Testing
Test Automation ensures we build the AGREED application
Exploratory Testing ensures we IMPROVE the application
Exploratory Testing
Exploratory Testing should be...
...done without knowledge of requirements.
...in time based sessions with specific focus area.
...outside the release lifecycle of the application.
Benefits
Continuous Regression testing gives instant feedback.
Continuously deploying to test servers tests deploy mechanism several
hundred times per release.
Always ready to push new release into UAT
We can get customer feedback as soon as value has been added
Test Driven Development
Everyone is responsible for Quality
Sustainable Quality
Tools Used
We love to talk about what tools are best and spend ages arguing about it.
We need to understand what we must do before we pick the tools.
Still which ever tool we pick it will be insufficient and crap.
Tools Used
Don't use one tool to rule them all.
Cucumber and Fitnesse are ok for system level testing
RESTAssured, JUnit or pure Groovy, Ruby or Python scripts are often better for
component level.
For Load testing one should use load testing tools such as JMeter
Continuous Delivery Process should execute tests through interface such as
maven or gradle so that it can be Test Tool agnostic.
Areas Not Covered
To do Continuous Delivery well we also have to...
Reproducible Test Environments
Production like Test Environments
“Non functional requirements” as functional requirements
A/B testing
Power of Metrics
and lots more...
Good Reading
Continuous Delivery by Jez Humble & Dave Farley
http://www.amazon.com/dp/0321601912?tag=contindelive-20
Thats it!
Feedback & Any questions you forgot to ask?
http://continuous-delivery-and-more.blogspot.se

Weitere ähnliche Inhalte

Was ist angesagt?

Dan Cuellar
Dan CuellarDan Cuellar
Dan CuellarCodeFest
 
Continuous Delivery Distilled
Continuous Delivery DistilledContinuous Delivery Distilled
Continuous Delivery DistilledMatt Callanan
 
I Don't Test Often ...
I Don't Test Often ...I Don't Test Often ...
I Don't Test Often ...Gareth Bowles
 
Continuous Deployment: The Dirty Details
Continuous Deployment: The Dirty DetailsContinuous Deployment: The Dirty Details
Continuous Deployment: The Dirty DetailsMike Brittain
 
Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...
Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...
Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...Matt Tesauro
 
Leandro Melendez - Switching Performance Left & Right
Leandro Melendez - Switching Performance Left & RightLeandro Melendez - Switching Performance Left & Right
Leandro Melendez - Switching Performance Left & RightNeotys_Partner
 
Introduction to Continuous Delivery (BBWorld/DevCon 2013)
Introduction to Continuous Delivery (BBWorld/DevCon 2013)Introduction to Continuous Delivery (BBWorld/DevCon 2013)
Introduction to Continuous Delivery (BBWorld/DevCon 2013)Mike McGarr
 
End-to-end performance testing, profiling, and analysis at Redis
End-to-end performance testing, profiling, and analysis at RedisEnd-to-end performance testing, profiling, and analysis at Redis
End-to-end performance testing, profiling, and analysis at RedisFilipe Oliveira
 
Continuous Integration
Continuous IntegrationContinuous Integration
Continuous Integrationdrluckyspin
 
Continuous everything
Continuous everythingContinuous everything
Continuous everythingTEST Huddle
 
QA in DevOps: Transformation thru Automation via Jenkins
QA in DevOps:  Transformation thru Automation via JenkinsQA in DevOps:  Transformation thru Automation via Jenkins
QA in DevOps: Transformation thru Automation via JenkinsTatyana Kravtsov
 
Continuous delivery - tools and techniques
Continuous delivery - tools and techniquesContinuous delivery - tools and techniques
Continuous delivery - tools and techniquesMike McGarr
 
JavaOne - Performance Focused DevOps to Improve Cont Delivery
JavaOne - Performance Focused DevOps to Improve Cont DeliveryJavaOne - Performance Focused DevOps to Improve Cont Delivery
JavaOne - Performance Focused DevOps to Improve Cont DeliveryAndreas Grabner
 
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...StarWest 2013 Performance is not an afterthought – make it a part of your Agi...
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...Andreas Grabner
 
AppSec Pipelines and Event based Security
AppSec Pipelines and Event based SecurityAppSec Pipelines and Event based Security
AppSec Pipelines and Event based SecurityMatt Tesauro
 
Where Testers & QA Fit in the Story of DevOps
Where Testers & QA Fit in the Story of DevOpsWhere Testers & QA Fit in the Story of DevOps
Where Testers & QA Fit in the Story of DevOpsQASymphony
 
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValueDevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValueRapidValue
 
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at EtsyPrinciples and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at EtsyMike Brittain
 
Continuous delivery applied
Continuous delivery appliedContinuous delivery applied
Continuous delivery appliedMike McGarr
 

Was ist angesagt? (20)

Dan Cuellar
Dan CuellarDan Cuellar
Dan Cuellar
 
Continuous Delivery Distilled
Continuous Delivery DistilledContinuous Delivery Distilled
Continuous Delivery Distilled
 
I Don't Test Often ...
I Don't Test Often ...I Don't Test Often ...
I Don't Test Often ...
 
Continuous Deployment: The Dirty Details
Continuous Deployment: The Dirty DetailsContinuous Deployment: The Dirty Details
Continuous Deployment: The Dirty Details
 
Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...
Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...
Making Continuous Security a Reality with OWASP’s AppSec Pipeline - Matt Tesa...
 
Leandro Melendez - Switching Performance Left & Right
Leandro Melendez - Switching Performance Left & RightLeandro Melendez - Switching Performance Left & Right
Leandro Melendez - Switching Performance Left & Right
 
Introduction to Continuous Delivery (BBWorld/DevCon 2013)
Introduction to Continuous Delivery (BBWorld/DevCon 2013)Introduction to Continuous Delivery (BBWorld/DevCon 2013)
Introduction to Continuous Delivery (BBWorld/DevCon 2013)
 
End-to-end performance testing, profiling, and analysis at Redis
End-to-end performance testing, profiling, and analysis at RedisEnd-to-end performance testing, profiling, and analysis at Redis
End-to-end performance testing, profiling, and analysis at Redis
 
DevOps Architecture Design
DevOps Architecture DesignDevOps Architecture Design
DevOps Architecture Design
 
Continuous Integration
Continuous IntegrationContinuous Integration
Continuous Integration
 
Continuous everything
Continuous everythingContinuous everything
Continuous everything
 
QA in DevOps: Transformation thru Automation via Jenkins
QA in DevOps:  Transformation thru Automation via JenkinsQA in DevOps:  Transformation thru Automation via Jenkins
QA in DevOps: Transformation thru Automation via Jenkins
 
Continuous delivery - tools and techniques
Continuous delivery - tools and techniquesContinuous delivery - tools and techniques
Continuous delivery - tools and techniques
 
JavaOne - Performance Focused DevOps to Improve Cont Delivery
JavaOne - Performance Focused DevOps to Improve Cont DeliveryJavaOne - Performance Focused DevOps to Improve Cont Delivery
JavaOne - Performance Focused DevOps to Improve Cont Delivery
 
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...StarWest 2013 Performance is not an afterthought – make it a part of your Agi...
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...
 
AppSec Pipelines and Event based Security
AppSec Pipelines and Event based SecurityAppSec Pipelines and Event based Security
AppSec Pipelines and Event based Security
 
Where Testers & QA Fit in the Story of DevOps
Where Testers & QA Fit in the Story of DevOpsWhere Testers & QA Fit in the Story of DevOps
Where Testers & QA Fit in the Story of DevOps
 
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValueDevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
DevOps Continuous Integration & Delivery - A Whitepaper by RapidValue
 
Principles and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at EtsyPrinciples and Practices in Continuous Deployment at Etsy
Principles and Practices in Continuous Deployment at Etsy
 
Continuous delivery applied
Continuous delivery appliedContinuous delivery applied
Continuous delivery applied
 

Andere mochten auch

Data science, DevOps, and drinks: The perfect combination
Data science, DevOps, and drinks: The perfect combinationData science, DevOps, and drinks: The perfect combination
Data science, DevOps, and drinks: The perfect combinationDonnie Berkholz
 
Continuous Testing for CTOs (Webinar Slides)
Continuous Testing for CTOs (Webinar Slides)Continuous Testing for CTOs (Webinar Slides)
Continuous Testing for CTOs (Webinar Slides)Rainforest QA
 
Web service testing_final.pptx
Web service testing_final.pptxWeb service testing_final.pptx
Web service testing_final.pptxvodqancr
 
DevOps Transformation - Another View
DevOps Transformation - Another ViewDevOps Transformation - Another View
DevOps Transformation - Another ViewAgron Fazliu
 
Ignite Talk: Help, we have no more time for testing!
Ignite Talk: Help, we have no more time for testing!Ignite Talk: Help, we have no more time for testing!
Ignite Talk: Help, we have no more time for testing!Dr. Alexander Schwartz
 
Time to REST: testing web services
Time to REST: testing web servicesTime to REST: testing web services
Time to REST: testing web servicesIurii Kutelmakh
 
Heleen Kuipers - presentatie reinventing organisations
Heleen Kuipers - presentatie reinventing organisationsHeleen Kuipers - presentatie reinventing organisations
Heleen Kuipers - presentatie reinventing organisationsForzesNL
 
Walery Lach, Marketing startup’ów – teoria a praktyka
Walery Lach, Marketing startup’ów – teoria a praktykaWalery Lach, Marketing startup’ów – teoria a praktyka
Walery Lach, Marketing startup’ów – teoria a praktykaSpeedUp Venture Capital Group
 
A/B testing. Content experiments in Google Analytics
A/B testing. Content experiments in Google AnalyticsA/B testing. Content experiments in Google Analytics
A/B testing. Content experiments in Google AnalyticsMichal Brys
 
Testing - Pawał Rutkowski
Testing - Pawał RutkowskiTesting - Pawał Rutkowski
Testing - Pawał Rutkowskiaulapolska
 
Creating a Java EE 7 Websocket Chat Application
Creating a Java EE 7 Websocket Chat ApplicationCreating a Java EE 7 Websocket Chat Application
Creating a Java EE 7 Websocket Chat ApplicationMicha Kops
 
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOpsDevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOpsSailaja Tennati
 
Testing RESTful web services with REST Assured
Testing RESTful web services with REST AssuredTesting RESTful web services with REST Assured
Testing RESTful web services with REST AssuredBas Dijkstra
 
Auto-scaled Concourse CI on AWS w/o BOSH
Auto-scaled Concourse CI on AWS w/o BOSHAuto-scaled Concourse CI on AWS w/o BOSH
Auto-scaled Concourse CI on AWS w/o BOSH佑介 九岡
 
Testy A/B na landing page'u
Testy A/B na landing page'uTesty A/B na landing page'u
Testy A/B na landing page'uLandingi
 

Andere mochten auch (20)

Reliability growth models
Reliability growth modelsReliability growth models
Reliability growth models
 
Data science, DevOps, and drinks: The perfect combination
Data science, DevOps, and drinks: The perfect combinationData science, DevOps, and drinks: The perfect combination
Data science, DevOps, and drinks: The perfect combination
 
Continuous Testing for CTOs (Webinar Slides)
Continuous Testing for CTOs (Webinar Slides)Continuous Testing for CTOs (Webinar Slides)
Continuous Testing for CTOs (Webinar Slides)
 
Web service testing_final.pptx
Web service testing_final.pptxWeb service testing_final.pptx
Web service testing_final.pptx
 
Autoscalable open API testing
Autoscalable open API testingAutoscalable open API testing
Autoscalable open API testing
 
DevOps Transformation - Another View
DevOps Transformation - Another ViewDevOps Transformation - Another View
DevOps Transformation - Another View
 
Coherent REST API design
Coherent REST API designCoherent REST API design
Coherent REST API design
 
Ignite Talk: Help, we have no more time for testing!
Ignite Talk: Help, we have no more time for testing!Ignite Talk: Help, we have no more time for testing!
Ignite Talk: Help, we have no more time for testing!
 
Time to REST: testing web services
Time to REST: testing web servicesTime to REST: testing web services
Time to REST: testing web services
 
Heleen Kuipers - presentatie reinventing organisations
Heleen Kuipers - presentatie reinventing organisationsHeleen Kuipers - presentatie reinventing organisations
Heleen Kuipers - presentatie reinventing organisations
 
Walery Lach, Marketing startup’ów – teoria a praktyka
Walery Lach, Marketing startup’ów – teoria a praktykaWalery Lach, Marketing startup’ów – teoria a praktyka
Walery Lach, Marketing startup’ów – teoria a praktyka
 
Przyszłość ma na imię Mobile – testowanie i automatyzacja testów aplikacji mo...
Przyszłość ma na imię Mobile – testowanie i automatyzacja testów aplikacji mo...Przyszłość ma na imię Mobile – testowanie i automatyzacja testów aplikacji mo...
Przyszłość ma na imię Mobile – testowanie i automatyzacja testów aplikacji mo...
 
A/B testing. Content experiments in Google Analytics
A/B testing. Content experiments in Google AnalyticsA/B testing. Content experiments in Google Analytics
A/B testing. Content experiments in Google Analytics
 
Testing - Pawał Rutkowski
Testing - Pawał RutkowskiTesting - Pawał Rutkowski
Testing - Pawał Rutkowski
 
Creating a Java EE 7 Websocket Chat Application
Creating a Java EE 7 Websocket Chat ApplicationCreating a Java EE 7 Websocket Chat Application
Creating a Java EE 7 Websocket Chat Application
 
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOpsDevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
DevOps Summit 2015 Presentation: Continuous Testing At the Speed of DevOps
 
Testing RESTful web services with REST Assured
Testing RESTful web services with REST AssuredTesting RESTful web services with REST Assured
Testing RESTful web services with REST Assured
 
Rest assured
Rest assuredRest assured
Rest assured
 
Auto-scaled Concourse CI on AWS w/o BOSH
Auto-scaled Concourse CI on AWS w/o BOSHAuto-scaled Concourse CI on AWS w/o BOSH
Auto-scaled Concourse CI on AWS w/o BOSH
 
Testy A/B na landing page'u
Testy A/B na landing page'uTesty A/B na landing page'u
Testy A/B na landing page'u
 

Ähnlich wie Continuous Delivery Testing @HiQ

Continuous delivery @ hi q
Continuous delivery @ hi qContinuous delivery @ hi q
Continuous delivery @ hi qTomas Riha
 
Continuous delivery its not about the technology, its about the people.
Continuous delivery its not about the technology, its about the people.Continuous delivery its not about the technology, its about the people.
Continuous delivery its not about the technology, its about the people.Tomas Riha
 
Continuous delivery its not about the technology, its about the people. @sats...
Continuous delivery its not about the technology, its about the people. @sats...Continuous delivery its not about the technology, its about the people. @sats...
Continuous delivery its not about the technology, its about the people. @sats...Tomas Riha
 
Continuous delivery its not about the technology, its about the people. @pipe...
Continuous delivery its not about the technology, its about the people. @pipe...Continuous delivery its not about the technology, its about the people. @pipe...
Continuous delivery its not about the technology, its about the people. @pipe...Tomas Riha
 
[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps
[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps
[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOpsKMS Technology
 
Continuous delivery @åf consult
Continuous delivery @åf consultContinuous delivery @åf consult
Continuous delivery @åf consultTomas Riha
 
product Qa workflow
product Qa workflowproduct Qa workflow
product Qa workflowtanvir afzal
 
Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys
Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code DeploysOur DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys
Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code DeploysDynatrace
 
Continuous delivery is more than dev ops
Continuous delivery is more than dev opsContinuous delivery is more than dev ops
Continuous delivery is more than dev opsAgile Montréal
 
Large-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in PracticeLarge-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in PracticeTechWell
 
Zero touch QA automation platform for DevOps
Zero touch QA automation platform for DevOpsZero touch QA automation platform for DevOps
Zero touch QA automation platform for DevOpsTaUB Solutions
 
Continuous Testing - The Final Frontier
Continuous Testing - The Final FrontierContinuous Testing - The Final Frontier
Continuous Testing - The Final FrontierJaco Greyling
 
Agile & DevOps - It's all about project success
Agile & DevOps - It's all about project successAgile & DevOps - It's all about project success
Agile & DevOps - It's all about project successAdam Stephensen
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsPerfecto Mobile
 
Life Has Not Been That Rosy With Agile : Rahul Sudame
Life Has Not Been That Rosy With Agile : Rahul SudameLife Has Not Been That Rosy With Agile : Rahul Sudame
Life Has Not Been That Rosy With Agile : Rahul SudameoGuild .
 

Ähnlich wie Continuous Delivery Testing @HiQ (20)

Continuous delivery @ hi q
Continuous delivery @ hi qContinuous delivery @ hi q
Continuous delivery @ hi q
 
Continuous delivery its not about the technology, its about the people.
Continuous delivery its not about the technology, its about the people.Continuous delivery its not about the technology, its about the people.
Continuous delivery its not about the technology, its about the people.
 
Continuous delivery its not about the technology, its about the people. @sats...
Continuous delivery its not about the technology, its about the people. @sats...Continuous delivery its not about the technology, its about the people. @sats...
Continuous delivery its not about the technology, its about the people. @sats...
 
Continuous delivery its not about the technology, its about the people. @pipe...
Continuous delivery its not about the technology, its about the people. @pipe...Continuous delivery its not about the technology, its about the people. @pipe...
Continuous delivery its not about the technology, its about the people. @pipe...
 
[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps
[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps
[Webinar] Test First, Fail Fast - Simplifying the Tester's Transition to DevOps
 
Future of QA
Future of QAFuture of QA
Future of QA
 
Futureofqa
FutureofqaFutureofqa
Futureofqa
 
Continuous delivery @åf consult
Continuous delivery @åf consultContinuous delivery @åf consult
Continuous delivery @åf consult
 
Agile testing
Agile testingAgile testing
Agile testing
 
product Qa workflow
product Qa workflowproduct Qa workflow
product Qa workflow
 
Continuous Delivery
Continuous DeliveryContinuous Delivery
Continuous Delivery
 
Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys
Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code DeploysOur DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys
Our DevOps Journey: 6 Month Waterfalls to 1 Hour Code Deploys
 
Continuous delivery is more than dev ops
Continuous delivery is more than dev opsContinuous delivery is more than dev ops
Continuous delivery is more than dev ops
 
Large-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in PracticeLarge-Scale Agile Test Automation Strategies in Practice
Large-Scale Agile Test Automation Strategies in Practice
 
Tce automation-d4
Tce automation-d4Tce automation-d4
Tce automation-d4
 
Zero touch QA automation platform for DevOps
Zero touch QA automation platform for DevOpsZero touch QA automation platform for DevOps
Zero touch QA automation platform for DevOps
 
Continuous Testing - The Final Frontier
Continuous Testing - The Final FrontierContinuous Testing - The Final Frontier
Continuous Testing - The Final Frontier
 
Agile & DevOps - It's all about project success
Agile & DevOps - It's all about project successAgile & DevOps - It's all about project success
Agile & DevOps - It's all about project success
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
 
Life Has Not Been That Rosy With Agile : Rahul Sudame
Life Has Not Been That Rosy With Agile : Rahul SudameLife Has Not Been That Rosy With Agile : Rahul Sudame
Life Has Not Been That Rosy With Agile : Rahul Sudame
 

Mehr von Tomas Riha

Driving change
Driving changeDriving change
Driving changeTomas Riha
 
DevOps - Its just Agile done right
DevOps - Its just Agile done rightDevOps - Its just Agile done right
DevOps - Its just Agile done rightTomas Riha
 
Test Automation
Test AutomationTest Automation
Test AutomationTomas Riha
 
Show me the money!
Show me the money!Show me the money!
Show me the money!Tomas Riha
 
Show me the money! - Draft
Show me the money! - DraftShow me the money! - Draft
Show me the money! - DraftTomas Riha
 
Into the cloud
Into the cloudInto the cloud
Into the cloudTomas Riha
 
Continuous delivery @ Diabol
Continuous delivery @ DiabolContinuous delivery @ Diabol
Continuous delivery @ DiabolTomas Riha
 
Scaling continuous delivery @ GeeCon 2014
Scaling continuous delivery @ GeeCon 2014Scaling continuous delivery @ GeeCon 2014
Scaling continuous delivery @ GeeCon 2014Tomas Riha
 

Mehr von Tomas Riha (9)

Driving change
Driving changeDriving change
Driving change
 
DevOps - Its just Agile done right
DevOps - Its just Agile done rightDevOps - Its just Agile done right
DevOps - Its just Agile done right
 
Test Automation
Test AutomationTest Automation
Test Automation
 
Show me the money!
Show me the money!Show me the money!
Show me the money!
 
Show me the money! - Draft
Show me the money! - DraftShow me the money! - Draft
Show me the money! - Draft
 
Dev ops
Dev opsDev ops
Dev ops
 
Into the cloud
Into the cloudInto the cloud
Into the cloud
 
Continuous delivery @ Diabol
Continuous delivery @ DiabolContinuous delivery @ Diabol
Continuous delivery @ Diabol
 
Scaling continuous delivery @ GeeCon 2014
Scaling continuous delivery @ GeeCon 2014Scaling continuous delivery @ GeeCon 2014
Scaling continuous delivery @ GeeCon 2014
 

Kürzlich hochgeladen

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 

Kürzlich hochgeladen (20)

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 

Continuous Delivery Testing @HiQ

  • 2. Tomas Riha Architect @ VGT/WirelessCar Passionate about creativity, change and improvement Horrible at following instructions and performing repetitive tasks MAJOR Project Liability mail: triha74@gmail.com twitter: @TomasRihaSE blog: continuous-delivery-and-more.blogspot.com
  • 3. Agenda Why do we want to do Continuous Delivery Intro to Continuous Delivery Principles of Continuous Delivery Look at a Pipe Test Automation for Continuous Delivery Test Driven Development and Continuous Delivery Exploratory Testing and Continuous Delivery Some words on Tooling Areas not covered
  • 4. Why do we want to do Continuous Delivery Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • 5. Why do we want to do Continuous Delivery Decreased lead time not only improves our time to market but also our ability to learn and improve through a faster feedback loop
  • 6. Why do we want to do Continuous Delivery As soon as we have added value to the application it needs to be in the hands of the customer or else we have just added cost.
  • 7. Why do we want to do Continuous Delivery Reduced risk through delivery of small changes.
  • 8. Are we really delivering as soon as possible? Pre Planning Dev Sys Test Reg Test Pre Planning Dev Sys Test Reg Test Scrummerfall happens because its hard for developers to have something deployable for the testers to test. System testing on something that has not been regression tested is fundamentally flawed. Cost of regression test tends to bloat sprint content. Sprint 2-4 weeks
  • 9. Continuous Delivery Build Release Deploy Test Prod Your application always builds, passes its unit tests, deploys and passes system tests.
  • 10. Continuous Integration/Delivery/Deploy Continuous Integration Your application always builds and passes its unit tests. Continuous Delivery Your application always builds, passes its unit tests, deploys and passes its system tests making it always ready to deploy to production through an automated process Continuous Deployment Your application automatically deploys to production through a fully automated process
  • 11. Continuously as in all the time? Yes! End of sprint isn't continuously. Weekly isn't continuously. Nightly isn't continuously. Every code commit is continuously!
  • 12. What Tests? Unit? Component? System? Yes! Unit Component System Integration Rollback Load Failover
  • 13. Principles of Continuous Delivery 1. The process MUST be repeatable, reliable and independent of key personnel
  • 14. 2. Automate everything Principles of Continuous Delivery
  • 15. 3. If something is difficult or painful, do it more often Principles of Continuous Delivery
  • 16. 4. Keep everything in source control AND release it Principles of Continuous Delivery
  • 17. 5. Done means released Principles of Continuous Delivery
  • 18. 6. Build Quality In Principles of Continuous Delivery
  • 19. 7. Fail Fast Principles of Continuous Delivery
  • 20. 8. Everybody has responsibility for the release process Principles of Continuous Delivery
  • 21. 9. Improve continuously Principles of Continuous Delivery
  • 22. The pipe - Build once! Build Release Build and Unit test then release it!
  • 23. The pipe - Release everything! Build Release Build Release Build Release Build Release Code DB Scripts Server Config Deploy script Feedback Feedback
  • 24. The pipe -Bundle released artifacts Build Release Build Release Build Release Build Release Assembly Feedback
  • 25. The pipe - Use Same Deploy Mechanism Function TestAssembly Deploy DB Server Function Test Function Test Pipe Status Feedback Feedback Feedback
  • 26. The pipe - Use Same Deploy Mechanism Deploy DB Server Pipe Status Server DB Server Server DB Server Server Deploy Pre Prod Prod UAT FeedbackFeedback Feedback
  • 27. Continuous Delivery Build Release Assemble Deploy Deploy PreProd/Prod Test Summary We build once Release everything Automate everything Fast feedback Automate all tests except UAT
  • 28. Obvious benefits Continuous Regression Testing gives instant feedback. Continuously deploying to test servers tests deploy mechanism several hundred times per release. Always ready to push new release into UAT We can get customer feedback as soon as value has been added
  • 29. Automate all tests! Wait a minute lets look at what we have been doing in the past.
  • 30. Testing in the past How its been done SYSTEM GUI
  • 31. Testing in the past What we have done 100s of manual tests that.. ... verify our requirements ... ensures bad input doesn't break system ... data is stored in the right place ... find bugs
  • 32. Testing in the past We have automated it! We have tried to automate the 100s of manual tests that.. ... verify our requirements ... ensures bad input doesn't break system ... data is stored in the right place ... find bugs We do it through instrumenting the GUI and verifying Database, GUI and System resources. With very bad results....
  • 33. Robust and Fast Test Automation Tests that execute 100s of times per day have to be... ... executable without any human interaction ... robust and give the same response every single time ... robust and have to survive refactoring ... fast and parallelizable as adding 10 tests should not slow down the feedback
  • 34. We need to change the way we look at Test On System Level we do verification of Use Case Acceptance Criterias. Requirement Verification. On Component Level we do testing of Component Functionality. Testing.
  • 35. We need to change the way we look at Architecture Remember “Build quality in”. We need to improve testability of our applications through... ...isolation of functionality into independent services with life cycles of their own. ...building interfaces for defined test points.
  • 37. Component Tests Internal Service Internal Service Internal Service Each Service Tested as Black Box in isolation and with high detail level
  • 38. GUI is a component GUI Service API Mock Impl GUI Test are much faster and robust if they test the GUI in isolation. High detail level.
  • 40. Continuous Delivery Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Assemble Release Deploy Test Component Pipe 100s of tests run in parallel testing System Pipe 10s Use Case Requirements Verified
  • 41. Test Driven Development Pre Planning Dev Sys Test Reg Test Pre Planning Dev Sys Test Reg Test Scrummerfall happens because its hard for developers to have something deployable for the testers to test. System testing on something that has not been regression tested is fundamentally flawed. Cost of regression test tends to bloat sprint content. Sprint 2-4 weeks
  • 42. Test Driven Development Pre Planning Dev Test Automation Reg Test Design Requirement Verification and Tests on multiple levels with the feature. Develop test automation in parallel with feature development. Cross functional pairing to improve quality of Tests, Test Architecture and Feature Testability Pre Planning Reg Test Reg Test Verification Verification
  • 43. Test Driven Development Who should automate the tests? Test Design - Done by Testers Test Automation - Done by ???? Coding - Done by Developers
  • 44. Test Driven Development Tests as Code Test Automation IS Code and needs to be treated as first class application. Test Architecture is super important and needs to be worked with in parallel with Application Testability Architecture.
  • 45. Test Driven Development Tests as Code Internal Service DSL Glue Code SpecificationText or Code based Specification Implementation of Specification Domain Specific Language packaging service testability
  • 46. Test Driven Development Internal Service DSL Glue Code Specification Give a valid user name and password when login is called then user is authorized boolean authorized = loginService.authorizeUser(“triha”,”cdrockz”); assert(authorized, true); public boolean authorizeUser(String userName, String password){ //Implementation of REST call to internal service return true; }
  • 47. Test Driven Development Tests as Code Specification & Glue Code IS the Test DSL is testability Test Automation is Code
  • 48. Test Driven Development Pre Planning Dev Test Automation Reg Test Specifications of Requirement Verification written during Pre-Planning. Specification of Component Tests written during Pre-Planning. Implementation of DSL and Glue Code in parallel with feature development. Implementation of Component Tests in parallel with feature development. Pre Planning Reg Test Reg Test Verification Verification
  • 49. Test Driven Development Tomorrow Agile Team Member Developers with Testing skills Testers with Development skills WILL BE SUPER HOT AND EARN SUPER NICE $$$$ Will be the only ones who get assignments
  • 50. Exploratory Testing Automate Everything! Yes! Everything that needs to be done in order to release a system into production. Yet we still need Exploratory testing.
  • 51. Exploratory Testing Has been used to cover up manual regression testing or worse A way to get away with non reproducible manual regression testing
  • 52. Exploratory Testing Test Automation ensures we build the AGREED application Exploratory Testing ensures we IMPROVE the application
  • 53. Exploratory Testing Exploratory Testing should be... ...done without knowledge of requirements. ...in time based sessions with specific focus area. ...outside the release lifecycle of the application.
  • 54. Benefits Continuous Regression testing gives instant feedback. Continuously deploying to test servers tests deploy mechanism several hundred times per release. Always ready to push new release into UAT We can get customer feedback as soon as value has been added Test Driven Development Everyone is responsible for Quality Sustainable Quality
  • 55. Tools Used We love to talk about what tools are best and spend ages arguing about it. We need to understand what we must do before we pick the tools. Still which ever tool we pick it will be insufficient and crap.
  • 56. Tools Used Don't use one tool to rule them all. Cucumber and Fitnesse are ok for system level testing RESTAssured, JUnit or pure Groovy, Ruby or Python scripts are often better for component level. For Load testing one should use load testing tools such as JMeter Continuous Delivery Process should execute tests through interface such as maven or gradle so that it can be Test Tool agnostic.
  • 57. Areas Not Covered To do Continuous Delivery well we also have to... Reproducible Test Environments Production like Test Environments “Non functional requirements” as functional requirements A/B testing Power of Metrics and lots more...
  • 58. Good Reading Continuous Delivery by Jez Humble & Dave Farley http://www.amazon.com/dp/0321601912?tag=contindelive-20
  • 59. Thats it! Feedback & Any questions you forgot to ask? http://continuous-delivery-and-more.blogspot.se