SlideShare ist ein Scribd-Unternehmen logo
1 von 17
Downloaden Sie, um offline zu lesen
Maturing your path toward DevOps with Continuous Testing
•
•
•
•
•
•
Twitter: @ek121268 (https://twitter.com/ek121268)
Blog: http://continuoustesting.blog
LinkedIn: https://www.linkedin.com/in/eran-kinsbruner-4b47a81/
5/24/2018 3© 2018, Perfecto Mobile Ltd. All Rights Reserved.
Continuous Testing is the process of executing
automated tests as part of the software
delivery pipeline in order to obtain feedback on
the business risks associated with a software
release candidate as rapidly as possible.
State of DevOps Report, 2018 - DORA
State of DevOps Report, 2018 - DORA
There Are Patterns for “Unstable” Test Automation
80% of issues have a pattern52% success rate
10% of devices,
causing 80% of lab
issues
Lab
25%
Orches
tration
25%
Scripts
& FW
50%
FAILURE REASON
Objects Codding Time Other
Scripts & FW issues
Device in use
No Device
Orchestration issues
Networking Stability Lock
Other
Lab issues
What’s
wrong
With my
Scripts
What’s wrong
With my Lab
What’s wrong
With my
Executions
5/24/2018 7© 2018, Perfecto Mobile Ltd. All Rights Reserved.
3
2
1
3
2
●
●
●
●
●
● ●
●
●
Maturing your path toward DevOps with Continuous Testing
Maturing your path toward DevOps with Continuous Testing
1. What’s the test engineer’s gut feeling 😊
2. Risk calculated as probability to occur and
impact to customers
3. Value – does the test provide new
information and, if failed, how much time to
fix?
4. Cost efficiency to develop – how long does
it take to develop and how easy is it to
script?
5. History of test – volume of historical
failures in related areas and frequency of
breaks
Source: Angie Jones
Insights into the CI
Pipeline
Risk/Focus Area
Mapping
Summary Report
List
Single Test Report
Visual Validations
Noise reduction through
error/failure-classification
• Pairing / Coaching
• Use the right object identification strategy
• Use the right test framework to work with
• Measure test efficiency within the CI
• Risk-based approach to test automation
• Continuous test data analysis and improvement
• How fast are testing activities moving, and what is slowing down these activities?
• Test flakiness
• Test duration
• % of automated vs. manual tests
• Application quality measurements
• # of escaped defects and in which areas
• MTTD – mean time to detection of defect
• Build quality
• Pipeline efficiency measurements
• # of user stories implemented per iteration
• Test automation as part of DoD across iterations
• Broken builds with categories
• CI length trending
• Lab availability and utilization
• Quality costs measurements
• Operational costs, lab availability issues
• Cost of hardware/software
• Costs of defects by severity and stage
Key ML Use Cases In Test Automation
• Recognize objects
• Transcribe speech – Accessibility
• Make quality related decisions based on data
• Identify Trends and/or Patterns
• Security use cases – Identify signatures e.g.
11/20/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Mindset and Workflow Changes in Test Automation
11/20/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved.
# Category
1 Test Authoring
2 Test Maintenance
3 Test Execution
4 Test Analysis
5 Tools Maturity
6 Available Integrations
7 Req. Test/Dev Skillset
8 Testing Env.
9 Testing Types
10 Supported App Types
Test Engineers/Developers Manual Testers (Developers?)
Traditional Test Automation
Define Manual Flows, BDD Style, etc.
Test Code & Reusable Functions (Java,
JS, etc.)
Define Objects, POM, use Object Spy
Structured Page Based Test Steps
Define Visual Validations and Assertions
Longer Time To Develop, Complex
Changes required pro-actively, SCM
Supported
Configure Env. (TestNG Data Provider),
Execute Locally/CI/Cloud
Flexible, OSS, Commercial
High, Including Guidelines, Doc’s
Plenty, OSS, Defect Management, etc.
Medium-High
IDE
API, Load, Functional
All Types
ML Based Test Automation
Record Test Flows (No Coding – in most cases)
Groups Reusable Functions
Objects Generated On-The-Fly (transparent to user)
Flow Based Scenarios
Part of Test Authoring Recording/Authoring
Shorter Time, Advanced Capabilities
Self-Healing/Correction Automatically handled/Object
Scoring/Local SCM Controlled
Configure Env., Execution Management Built-In,
Locally/CI/Cloud
Relies on ML/AI Tool Vendor in Most Cases
Emerging Technology, Web More Advanced Than
Mobile
Most capabilities built-in, some exists
Low-Medium
ML UI
Mostly Functional (E2E) & API
Mostly Web
Maturing your path toward DevOps with Continuous Testing
Maturing your path toward DevOps with Continuous Testing

Más contenido relacionado

Was ist angesagt?

How to Scale Digital App Testing With Jenkins & Automation You Can Trust
How to Scale Digital App Testing With Jenkins & Automation You Can TrustHow to Scale Digital App Testing With Jenkins & Automation You Can Trust
How to Scale Digital App Testing With Jenkins & Automation You Can TrustPerfecto by Perforce
 
Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...
Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...
Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...SmartBear
 
Automated Code Reviews with AI and ML - DevOps Next
Automated Code Reviews with AI and ML - DevOps NextAutomated Code Reviews with AI and ML - DevOps Next
Automated Code Reviews with AI and ML - DevOps NextPerfecto by Perforce
 
ESLint Plugin for UI Tests
ESLint Plugin for UI TestsESLint Plugin for UI Tests
ESLint Plugin for UI TestsApplitools
 
Use Automation to Assist -Not Replace- Manual Testing
Use Automation to Assist -Not Replace- Manual TestingUse Automation to Assist -Not Replace- Manual Testing
Use Automation to Assist -Not Replace- Manual TestingSmartBear
 
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...Applitools
 
ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...
ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...
ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...Agile Testing Alliance
 
Leveraging AI and ML in Test Management Systems - DevOps Next
Leveraging AI and ML in Test Management Systems - DevOps NextLeveraging AI and ML in Test Management Systems - DevOps Next
Leveraging AI and ML in Test Management Systems - DevOps NextPerfecto by Perforce
 
Automated Visual Testing at Scale : Real-life Example from Dow Jones
Automated Visual Testing at Scale : Real-life Example from Dow JonesAutomated Visual Testing at Scale : Real-life Example from Dow Jones
Automated Visual Testing at Scale : Real-life Example from Dow JonesApplitools
 
Understanding and Executing on API Developer Experience
Understanding and Executing on API Developer ExperienceUnderstanding and Executing on API Developer Experience
Understanding and Executing on API Developer ExperienceSmartBear
 
Advancing the State of The Art in AI and Testing - DevOps Next
Advancing the State of The Art in AI and Testing - DevOps NextAdvancing the State of The Art in AI and Testing - DevOps Next
Advancing the State of The Art in AI and Testing - DevOps NextPerfecto by Perforce
 
"Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry""Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry"Applitools
 
The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021DevOps.com
 
ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...
ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...
ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...Agile Testing Alliance
 
ATAGTR2017 SPEAKING EYE for differently abled people to see the web content
ATAGTR2017 SPEAKING EYE for differently abled people to see the web contentATAGTR2017 SPEAKING EYE for differently abled people to see the web content
ATAGTR2017 SPEAKING EYE for differently abled people to see the web contentAgile Testing Alliance
 

Was ist angesagt? (20)

ATAGTR2017 Testing in DevOps Culture
ATAGTR2017 Testing in DevOps CultureATAGTR2017 Testing in DevOps Culture
ATAGTR2017 Testing in DevOps Culture
 
ATAGTR2017 Wearable App Testing
ATAGTR2017 Wearable App TestingATAGTR2017 Wearable App Testing
ATAGTR2017 Wearable App Testing
 
How to Scale Digital App Testing With Jenkins & Automation You Can Trust
How to Scale Digital App Testing With Jenkins & Automation You Can TrustHow to Scale Digital App Testing With Jenkins & Automation You Can Trust
How to Scale Digital App Testing With Jenkins & Automation You Can Trust
 
Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...
Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...
Definition Driven API Development: How OAS & Swagger Help Teams Streamline Th...
 
Automated Code Reviews with AI and ML - DevOps Next
Automated Code Reviews with AI and ML - DevOps NextAutomated Code Reviews with AI and ML - DevOps Next
Automated Code Reviews with AI and ML - DevOps Next
 
ESLint Plugin for UI Tests
ESLint Plugin for UI TestsESLint Plugin for UI Tests
ESLint Plugin for UI Tests
 
Use Automation to Assist -Not Replace- Manual Testing
Use Automation to Assist -Not Replace- Manual TestingUse Automation to Assist -Not Replace- Manual Testing
Use Automation to Assist -Not Replace- Manual Testing
 
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
 
BDD along with Continuous Integration
BDD along with Continuous IntegrationBDD along with Continuous Integration
BDD along with Continuous Integration
 
ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...
ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...
ATAGTR2017 Cost-effective Security Testing Approaches for Web, Mobile & Enter...
 
Leveraging AI and ML in Test Management Systems - DevOps Next
Leveraging AI and ML in Test Management Systems - DevOps NextLeveraging AI and ML in Test Management Systems - DevOps Next
Leveraging AI and ML in Test Management Systems - DevOps Next
 
Automated Visual Testing at Scale : Real-life Example from Dow Jones
Automated Visual Testing at Scale : Real-life Example from Dow JonesAutomated Visual Testing at Scale : Real-life Example from Dow Jones
Automated Visual Testing at Scale : Real-life Example from Dow Jones
 
Understanding and Executing on API Developer Experience
Understanding and Executing on API Developer ExperienceUnderstanding and Executing on API Developer Experience
Understanding and Executing on API Developer Experience
 
Advancing the State of The Art in AI and Testing - DevOps Next
Advancing the State of The Art in AI and Testing - DevOps NextAdvancing the State of The Art in AI and Testing - DevOps Next
Advancing the State of The Art in AI and Testing - DevOps Next
 
"Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry""Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry"
 
Model-Based Testing for Cypress
Model-Based Testing for CypressModel-Based Testing for Cypress
Model-Based Testing for Cypress
 
The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021The Evolving Role of the Developer in 2021
The Evolving Role of the Developer in 2021
 
ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...
ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...
ATAGTR2017 Static and dynamic code analysis for mobile applications - Act ear...
 
ATAGTR2017 SPEAKING EYE for differently abled people to see the web content
ATAGTR2017 SPEAKING EYE for differently abled people to see the web contentATAGTR2017 SPEAKING EYE for differently abled people to see the web content
ATAGTR2017 SPEAKING EYE for differently abled people to see the web content
 
ATAGTR2017 ChatBot
ATAGTR2017 ChatBotATAGTR2017 ChatBot
ATAGTR2017 ChatBot
 

Ähnlich wie Maturing your path toward DevOps with Continuous Testing

Creating a successful continuous testing environment by Eran Kinsbruner
Creating a successful continuous testing environment by Eran KinsbrunerCreating a successful continuous testing environment by Eran Kinsbruner
Creating a successful continuous testing environment by Eran KinsbrunerQA or the Highway
 
Creating a Successful Continuous Testing Environment
Creating a Successful Continuous Testing EnvironmentCreating a Successful Continuous Testing Environment
Creating a Successful Continuous Testing EnvironmentPerfecto by Perforce
 
Amalgamation of BDD, parallel execution and mobile automation
Amalgamation of BDD, parallel execution and mobile automationAmalgamation of BDD, parallel execution and mobile automation
Amalgamation of BDD, parallel execution and mobile automationAgile Testing Alliance
 
5 Stages of Digital Quality Maturity
5 Stages of Digital Quality Maturity5 Stages of Digital Quality Maturity
5 Stages of Digital Quality MaturityPerfecto Mobile
 
Aligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development PracticesAligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development PracticesAspire Systems
 
End the Guesswork: How to Test the Right Devices, Platforms & User Conditions
End the Guesswork: How to Test the Right Devices, Platforms & User ConditionsEnd the Guesswork: How to Test the Right Devices, Platforms & User Conditions
End the Guesswork: How to Test the Right Devices, Platforms & User ConditionsPerfecto by Perforce
 
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...Curiosity Software Ireland
 
How to Avoid Continuously Delivering Faulty Software
How to Avoid Continuously Delivering Faulty SoftwareHow to Avoid Continuously Delivering Faulty Software
How to Avoid Continuously Delivering Faulty SoftwarePerforce
 
Shailaja_SoftwareEngineerQA_4.9Exp
Shailaja_SoftwareEngineerQA_4.9ExpShailaja_SoftwareEngineerQA_4.9Exp
Shailaja_SoftwareEngineerQA_4.9ExpShailaja Chundru
 
Top 5 Automation Challenges Webinar
Top 5 Automation Challenges WebinarTop 5 Automation Challenges Webinar
Top 5 Automation Challenges WebinarPerfecto by Perforce
 
How Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for itHow Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for itLizzy Guido (she/her)
 
Mobile App Quality Roadmap for DevTest Teams
Mobile App Quality Roadmap for DevTest TeamsMobile App Quality Roadmap for DevTest Teams
Mobile App Quality Roadmap for DevTest TeamsPerfecto by Perforce
 
Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster Curiosity Software Ireland
 
Integrating AI in software quality in absence of a well-defined requirements
Integrating AI in software quality in absence of a well-defined requirementsIntegrating AI in software quality in absence of a well-defined requirements
Integrating AI in software quality in absence of a well-defined requirementsNagarro
 
Kazi Hossein RESUME1
Kazi Hossein RESUME1Kazi Hossein RESUME1
Kazi Hossein RESUME1kazi hossain
 

Ähnlich wie Maturing your path toward DevOps with Continuous Testing (20)

Creating a successful continuous testing environment by Eran Kinsbruner
Creating a successful continuous testing environment by Eran KinsbrunerCreating a successful continuous testing environment by Eran Kinsbruner
Creating a successful continuous testing environment by Eran Kinsbruner
 
Creating a Successful Continuous Testing Environment
Creating a Successful Continuous Testing EnvironmentCreating a Successful Continuous Testing Environment
Creating a Successful Continuous Testing Environment
 
Amalgamation of BDD, parallel execution and mobile automation
Amalgamation of BDD, parallel execution and mobile automationAmalgamation of BDD, parallel execution and mobile automation
Amalgamation of BDD, parallel execution and mobile automation
 
5 Stages of Digital Quality Maturity
5 Stages of Digital Quality Maturity5 Stages of Digital Quality Maturity
5 Stages of Digital Quality Maturity
 
Aligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development PracticesAligning Software Testing With Modern Age Development Practices
Aligning Software Testing With Modern Age Development Practices
 
Gcs day1
Gcs day1Gcs day1
Gcs day1
 
End the Guesswork: How to Test the Right Devices, Platforms & User Conditions
End the Guesswork: How to Test the Right Devices, Platforms & User ConditionsEnd the Guesswork: How to Test the Right Devices, Platforms & User Conditions
End the Guesswork: How to Test the Right Devices, Platforms & User Conditions
 
Amita_Kashyap1_CV
Amita_Kashyap1_CVAmita_Kashyap1_CV
Amita_Kashyap1_CV
 
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
 
Software testing
Software testingSoftware testing
Software testing
 
How to Avoid Continuously Delivering Faulty Software
How to Avoid Continuously Delivering Faulty SoftwareHow to Avoid Continuously Delivering Faulty Software
How to Avoid Continuously Delivering Faulty Software
 
Shailaja_SoftwareEngineerQA_4.9Exp
Shailaja_SoftwareEngineerQA_4.9ExpShailaja_SoftwareEngineerQA_4.9Exp
Shailaja_SoftwareEngineerQA_4.9Exp
 
Top 5 Automation Challenges Webinar
Top 5 Automation Challenges WebinarTop 5 Automation Challenges Webinar
Top 5 Automation Challenges Webinar
 
How Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for itHow Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for it
 
Mobile App Quality Roadmap for DevTest Teams
Mobile App Quality Roadmap for DevTest TeamsMobile App Quality Roadmap for DevTest Teams
Mobile App Quality Roadmap for DevTest Teams
 
My Profile
My ProfileMy Profile
My Profile
 
Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster
 
Integrating AI in software quality in absence of a well-defined requirements
Integrating AI in software quality in absence of a well-defined requirementsIntegrating AI in software quality in absence of a well-defined requirements
Integrating AI in software quality in absence of a well-defined requirements
 
9 Yrs Manual and Selenium Testing Profile
9 Yrs Manual and Selenium Testing Profile9 Yrs Manual and Selenium Testing Profile
9 Yrs Manual and Selenium Testing Profile
 
Kazi Hossein RESUME1
Kazi Hossein RESUME1Kazi Hossein RESUME1
Kazi Hossein RESUME1
 

Mehr von Perfecto Mobile

QA or the Highway 2022.pptx
QA or the Highway 2022.pptxQA or the Highway 2022.pptx
QA or the Highway 2022.pptxPerfecto Mobile
 
Proven strategy for testing pw as aus-agile_testers
Proven strategy for testing pw as aus-agile_testersProven strategy for testing pw as aus-agile_testers
Proven strategy for testing pw as aus-agile_testersPerfecto Mobile
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsPerfecto Mobile
 
Selenium conf india pwa 2018
Selenium conf india pwa 2018Selenium conf india pwa 2018
Selenium conf india pwa 2018Perfecto Mobile
 
Online testconf event rwd and pwa 2018
Online testconf event rwd and pwa 2018Online testconf event rwd and pwa 2018
Online testconf event rwd and pwa 2018Perfecto Mobile
 
Optimizing Test Coverage throughout the DevOps Pipeline
Optimizing Test Coverage throughout the DevOps PipelineOptimizing Test Coverage throughout the DevOps Pipeline
Optimizing Test Coverage throughout the DevOps PipelinePerfecto Mobile
 
Quest2018 erank optimize test automation
Quest2018 erank optimize test automationQuest2018 erank optimize test automation
Quest2018 erank optimize test automationPerfecto Mobile
 
Mastering BDD - Eran Kinsbruner Workshop Quest 2018
Mastering BDD - Eran Kinsbruner Workshop Quest 2018Mastering BDD - Eran Kinsbruner Workshop Quest 2018
Mastering BDD - Eran Kinsbruner Workshop Quest 2018Perfecto Mobile
 
Mapping mobileandweblandscape motb
Mapping mobileandweblandscape motbMapping mobileandweblandscape motb
Mapping mobileandweblandscape motbPerfecto Mobile
 
Testing Next Generation Digital Interfaces
Testing Next Generation Digital InterfacesTesting Next Generation Digital Interfaces
Testing Next Generation Digital InterfacesPerfecto Mobile
 
Raleigh User Forum Presentation
Raleigh User Forum PresentationRaleigh User Forum Presentation
Raleigh User Forum PresentationPerfecto Mobile
 
Automation challenges - 121 Test Automation Event boston
Automation challenges - 121 Test Automation Event bostonAutomation challenges - 121 Test Automation Event boston
Automation challenges - 121 Test Automation Event bostonPerfecto Mobile
 
Boston meetup blaze_meter_feb2017
Boston meetup blaze_meter_feb2017Boston meetup blaze_meter_feb2017
Boston meetup blaze_meter_feb2017Perfecto Mobile
 
7 Keys for Unattended Test AUtomation webinar deck
7 Keys for Unattended Test AUtomation webinar deck7 Keys for Unattended Test AUtomation webinar deck
7 Keys for Unattended Test AUtomation webinar deckPerfecto Mobile
 
iOS9 Launch - Mobile Dev & Test Implications
iOS9 Launch - Mobile Dev & Test ImplicationsiOS9 Launch - Mobile Dev & Test Implications
iOS9 Launch - Mobile Dev & Test ImplicationsPerfecto Mobile
 
Continuous Quality For a 5 Star Mobile Apps Delivery
Continuous Quality For a 5 Star Mobile Apps DeliveryContinuous Quality For a 5 Star Mobile Apps Delivery
Continuous Quality For a 5 Star Mobile Apps DeliveryPerfecto Mobile
 
Mobile Test Coverage- Israel 4th meetup
Mobile Test Coverage- Israel 4th meetupMobile Test Coverage- Israel 4th meetup
Mobile Test Coverage- Israel 4th meetupPerfecto Mobile
 
Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star eastPerfecto Mobile
 

Mehr von Perfecto Mobile (20)

QA or the Highway 2022.pptx
QA or the Highway 2022.pptxQA or the Highway 2022.pptx
QA or the Highway 2022.pptx
 
Proven strategy for testing pw as aus-agile_testers
Proven strategy for testing pw as aus-agile_testersProven strategy for testing pw as aus-agile_testers
Proven strategy for testing pw as aus-agile_testers
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
 
Selenium conf india pwa 2018
Selenium conf india pwa 2018Selenium conf india pwa 2018
Selenium conf india pwa 2018
 
Online testconf event rwd and pwa 2018
Online testconf event rwd and pwa 2018Online testconf event rwd and pwa 2018
Online testconf event rwd and pwa 2018
 
Optimizing Test Coverage throughout the DevOps Pipeline
Optimizing Test Coverage throughout the DevOps PipelineOptimizing Test Coverage throughout the DevOps Pipeline
Optimizing Test Coverage throughout the DevOps Pipeline
 
Cross browser testing
Cross browser testingCross browser testing
Cross browser testing
 
Quest2018 erank optimize test automation
Quest2018 erank optimize test automationQuest2018 erank optimize test automation
Quest2018 erank optimize test automation
 
Mastering BDD - Eran Kinsbruner Workshop Quest 2018
Mastering BDD - Eran Kinsbruner Workshop Quest 2018Mastering BDD - Eran Kinsbruner Workshop Quest 2018
Mastering BDD - Eran Kinsbruner Workshop Quest 2018
 
Mapping mobileandweblandscape motb
Mapping mobileandweblandscape motbMapping mobileandweblandscape motb
Mapping mobileandweblandscape motb
 
Testing Next Generation Digital Interfaces
Testing Next Generation Digital InterfacesTesting Next Generation Digital Interfaces
Testing Next Generation Digital Interfaces
 
Raleigh User Forum Presentation
Raleigh User Forum PresentationRaleigh User Forum Presentation
Raleigh User Forum Presentation
 
Automation challenges - 121 Test Automation Event boston
Automation challenges - 121 Test Automation Event bostonAutomation challenges - 121 Test Automation Event boston
Automation challenges - 121 Test Automation Event boston
 
Boston meetup blaze_meter_feb2017
Boston meetup blaze_meter_feb2017Boston meetup blaze_meter_feb2017
Boston meetup blaze_meter_feb2017
 
7 Keys for Unattended Test AUtomation webinar deck
7 Keys for Unattended Test AUtomation webinar deck7 Keys for Unattended Test AUtomation webinar deck
7 Keys for Unattended Test AUtomation webinar deck
 
iOS9 Launch - Mobile Dev & Test Implications
iOS9 Launch - Mobile Dev & Test ImplicationsiOS9 Launch - Mobile Dev & Test Implications
iOS9 Launch - Mobile Dev & Test Implications
 
Continuous Quality For a 5 Star Mobile Apps Delivery
Continuous Quality For a 5 Star Mobile Apps DeliveryContinuous Quality For a 5 Star Mobile Apps Delivery
Continuous Quality For a 5 Star Mobile Apps Delivery
 
Mobile Test Coverage- Israel 4th meetup
Mobile Test Coverage- Israel 4th meetupMobile Test Coverage- Israel 4th meetup
Mobile Test Coverage- Israel 4th meetup
 
Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star east
 
Wearables meetup
Wearables meetupWearables meetup
Wearables meetup
 

Último

Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4DianaGray10
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechProduct School
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxNeo4j
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applicationsnooralam814309
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Muhammad Tiham Siddiqui
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch TuesdayIvanti
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 

Último (20)

Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
Graphene Quantum Dots-Based Composites for Biomedical Applications
Graphene Quantum Dots-Based Composites for  Biomedical ApplicationsGraphene Quantum Dots-Based Composites for  Biomedical Applications
Graphene Quantum Dots-Based Composites for Biomedical Applications
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch Tuesday
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 

Maturing your path toward DevOps with Continuous Testing

  • 2. • • • • • • Twitter: @ek121268 (https://twitter.com/ek121268) Blog: http://continuoustesting.blog LinkedIn: https://www.linkedin.com/in/eran-kinsbruner-4b47a81/
  • 3. 5/24/2018 3© 2018, Perfecto Mobile Ltd. All Rights Reserved. Continuous Testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.
  • 4. State of DevOps Report, 2018 - DORA
  • 5. State of DevOps Report, 2018 - DORA
  • 6. There Are Patterns for “Unstable” Test Automation 80% of issues have a pattern52% success rate 10% of devices, causing 80% of lab issues Lab 25% Orches tration 25% Scripts & FW 50% FAILURE REASON Objects Codding Time Other Scripts & FW issues Device in use No Device Orchestration issues Networking Stability Lock Other Lab issues What’s wrong With my Scripts What’s wrong With my Lab What’s wrong With my Executions
  • 7. 5/24/2018 7© 2018, Perfecto Mobile Ltd. All Rights Reserved. 3 2 1 3 2 ● ● ● ● ● ● ● ● ●
  • 10. 1. What’s the test engineer’s gut feeling 😊 2. Risk calculated as probability to occur and impact to customers 3. Value – does the test provide new information and, if failed, how much time to fix? 4. Cost efficiency to develop – how long does it take to develop and how easy is it to script? 5. History of test – volume of historical failures in related areas and frequency of breaks Source: Angie Jones
  • 11. Insights into the CI Pipeline Risk/Focus Area Mapping Summary Report List Single Test Report Visual Validations Noise reduction through error/failure-classification
  • 12. • Pairing / Coaching • Use the right object identification strategy • Use the right test framework to work with • Measure test efficiency within the CI • Risk-based approach to test automation • Continuous test data analysis and improvement
  • 13. • How fast are testing activities moving, and what is slowing down these activities? • Test flakiness • Test duration • % of automated vs. manual tests • Application quality measurements • # of escaped defects and in which areas • MTTD – mean time to detection of defect • Build quality • Pipeline efficiency measurements • # of user stories implemented per iteration • Test automation as part of DoD across iterations • Broken builds with categories • CI length trending • Lab availability and utilization • Quality costs measurements • Operational costs, lab availability issues • Cost of hardware/software • Costs of defects by severity and stage
  • 14. Key ML Use Cases In Test Automation • Recognize objects • Transcribe speech – Accessibility • Make quality related decisions based on data • Identify Trends and/or Patterns • Security use cases – Identify signatures e.g. 11/20/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 15. Mindset and Workflow Changes in Test Automation 11/20/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved. # Category 1 Test Authoring 2 Test Maintenance 3 Test Execution 4 Test Analysis 5 Tools Maturity 6 Available Integrations 7 Req. Test/Dev Skillset 8 Testing Env. 9 Testing Types 10 Supported App Types Test Engineers/Developers Manual Testers (Developers?) Traditional Test Automation Define Manual Flows, BDD Style, etc. Test Code & Reusable Functions (Java, JS, etc.) Define Objects, POM, use Object Spy Structured Page Based Test Steps Define Visual Validations and Assertions Longer Time To Develop, Complex Changes required pro-actively, SCM Supported Configure Env. (TestNG Data Provider), Execute Locally/CI/Cloud Flexible, OSS, Commercial High, Including Guidelines, Doc’s Plenty, OSS, Defect Management, etc. Medium-High IDE API, Load, Functional All Types ML Based Test Automation Record Test Flows (No Coding – in most cases) Groups Reusable Functions Objects Generated On-The-Fly (transparent to user) Flow Based Scenarios Part of Test Authoring Recording/Authoring Shorter Time, Advanced Capabilities Self-Healing/Correction Automatically handled/Object Scoring/Local SCM Controlled Configure Env., Execution Management Built-In, Locally/CI/Cloud Relies on ML/AI Tool Vendor in Most Cases Emerging Technology, Web More Advanced Than Mobile Most capabilities built-in, some exists Low-Medium ML UI Mostly Functional (E2E) & API Mostly Web