SlideShare ist ein Scribd-Unternehmen logo
1 von 23
Want Higher Quality Software
from Your Agile Team?
Cross-Functional Peer Review Works
Mark Hammer
When you create or receive
a test plan, who sees it – and how?
Product Mgmt

Development

Testing

User stories

Code

Test Plans
Product Mgmt

Development
User stories
Code
Test Plans

Testing
Cross-Functional Peer Review
Requirements

Discussion

Design

Review

Architecture

Review
Requirements

Discussion

Design

Review

Architecture

Review

Code

N/A
Hapless
Developer

Version
Control

Reviewers
Types of Code Review






Over-the-Shoulder
Email
Pair Programming
Formal Inspection Meetings
Tool
Largest Peer Code Review Study
•

•
•
•
•
•

Objectives:
– lightweight vs. formal inspections
– What constitutes an effective review?
10-month case study at Cisco
Cisco MeetingPlace product, teleconferencing solution
3.2 million lines of code
2500 reviews
50 developers
Defects Found

60-90 minutes max

Time (minutes)
Go slow: 200-500 LOC/hour
Defect Density vs. Inspection Rate

Defect Density (defects/kLOC)

150
125
100
75
50
25
0
0

200

400

600

800

1000

Review Inspection Rate (LOC/hour)

1200

1400
Not too much: 200-400 LOC
Defect Density vs. LOC

Defect Density (defects/kLOC)

200

150

100

50

0
0

200

400

600

LOC under Review

800

1000
Industry Metrics
Measure

Industry
Average

High
Performance
Teams

Net Promoter Score

20%

> 70%

% defects of total injected found by customer

15%

< 2%

% effort spent in finding and fixing defects

50%

< 10%

% effort for post-release support

30%

< 5%

Unit test code coverage

Varies

> 80%

Post release defect density

7.5
defects/KLOC

< 0.5
defects/KLOC
Measure

Industry
Average

High
Performance
Teams

Net Promoter Score

20%

> 70%

% defects of total injected found by customer

15%

< 2%

% effort spent in finding and fixing defects

50%

< 10%

% effort for post-release support

30%

< 5%

Unit test code coverage

Varies

> 80%

Post release defect density

7.5
defects/KLOC

< 0.5
defects/KLOC

 Bugs found in development are 8-12X less expensive to fix than those

found in testing phase
 And 30-100X less expensive than bugs that reach customers
Case Study:
Large National Insurance Company
• 2011: 350 developers
• 2013: 650 team members
• User stories are shared in Word format with entire
team
• Design documents are shared in Powerpoint with
entire team
• Code is shared with entire team
• Test cases are shared in Excel format with entire team
Benefits: Cross-Functional Peer
Review
• Every member of the extended development team
knows what’s happening
• Problems with user stories, code, and test plans are
found faster
• Developers are forced to write readable code
• Optimization methods/tricks/productive programs
spread faster
• Teams can iterate from story to code to test plan
• It's Agile
• It’s fun
Product Mgmt

Development
User stories
Code
Test Plans

Testing
“The simple fact of knowing your work will be
reviewed by others means you’ll do it better.”
Questions?
Comments?

Mark Hammer

Weitere ähnliche Inhalte

Was ist angesagt?

Test Automation and Continuous Integration
Test Automation and Continuous Integration Test Automation and Continuous Integration
Test Automation and Continuous Integration
TestCampRO
 
Automated Unit Testing
Automated Unit Testing Automated Unit Testing
Automated Unit Testing
Simon Boorsma
 
Becoming a better programmer - unit testing
Becoming a better programmer - unit testingBecoming a better programmer - unit testing
Becoming a better programmer - unit testing
Duy Tan Geek
 
Ady beleanu automate-theprocessdelivery
Ady beleanu   automate-theprocessdeliveryAdy beleanu   automate-theprocessdelivery
Ady beleanu automate-theprocessdelivery
Romania Testing
 
Testing Frameworks And Methodologies
Testing Frameworks And MethodologiesTesting Frameworks And Methodologies
Testing Frameworks And Methodologies
Steven Cahill
 

Was ist angesagt? (20)

Test Automation and Continuous Integration
Test Automation and Continuous Integration Test Automation and Continuous Integration
Test Automation and Continuous Integration
 
Test Design: Scaling UI Automation from the Ground Up
Test Design: Scaling UI Automation from the Ground UpTest Design: Scaling UI Automation from the Ground Up
Test Design: Scaling UI Automation from the Ground Up
 
Automated Unit Testing
Automated Unit Testing Automated Unit Testing
Automated Unit Testing
 
Visual Studio 2010 Testing for Developers
Visual Studio 2010 Testing for DevelopersVisual Studio 2010 Testing for Developers
Visual Studio 2010 Testing for Developers
 
What is DevOps?
What is DevOps?What is DevOps?
What is DevOps?
 
BDD along with Continuous Integration
BDD along with Continuous IntegrationBDD along with Continuous Integration
BDD along with Continuous Integration
 
Amalgamation of BDD, parallel execution and mobile automation
Amalgamation of BDD, parallel execution and mobile automationAmalgamation of BDD, parallel execution and mobile automation
Amalgamation of BDD, parallel execution and mobile automation
 
Automation qa
Automation qaAutomation qa
Automation qa
 
Unit Testing and Tools - ADNUG
Unit Testing and Tools - ADNUGUnit Testing and Tools - ADNUG
Unit Testing and Tools - ADNUG
 
Testing Frameworks
Testing FrameworksTesting Frameworks
Testing Frameworks
 
Unit Testing and Tools
Unit Testing and ToolsUnit Testing and Tools
Unit Testing and Tools
 
Role of Test Automation in Modern Software Delivery Pipelines
Role of Test Automation in Modern Software Delivery PipelinesRole of Test Automation in Modern Software Delivery Pipelines
Role of Test Automation in Modern Software Delivery Pipelines
 
Introduction to Gauge
Introduction to GaugeIntroduction to Gauge
Introduction to Gauge
 
ATAGTR2017 Differentiation using Testing Tools and Automation in the BFS COTS...
ATAGTR2017 Differentiation using Testing Tools and Automation in the BFS COTS...ATAGTR2017 Differentiation using Testing Tools and Automation in the BFS COTS...
ATAGTR2017 Differentiation using Testing Tools and Automation in the BFS COTS...
 
Becoming a better programmer - unit testing
Becoming a better programmer - unit testingBecoming a better programmer - unit testing
Becoming a better programmer - unit testing
 
Myths and reality about software testing
Myths and reality about software testingMyths and reality about software testing
Myths and reality about software testing
 
Ady beleanu automate-theprocessdelivery
Ady beleanu   automate-theprocessdeliveryAdy beleanu   automate-theprocessdelivery
Ady beleanu automate-theprocessdelivery
 
Testing Frameworks And Methodologies
Testing Frameworks And MethodologiesTesting Frameworks And Methodologies
Testing Frameworks And Methodologies
 
TGT#16 - Pain of test automatization in PAAS/SAAS solutions - Krzysztof Lembi...
TGT#16 - Pain of test automatization in PAAS/SAAS solutions - Krzysztof Lembi...TGT#16 - Pain of test automatization in PAAS/SAAS solutions - Krzysztof Lembi...
TGT#16 - Pain of test automatization in PAAS/SAAS solutions - Krzysztof Lembi...
 
Selenium training eduxfactor
Selenium training   eduxfactorSelenium training   eduxfactor
Selenium training eduxfactor
 

Ähnlich wie Cross functional peer review preso 10-01-2013

Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity Software Ireland
 
Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster
Curiosity Software Ireland
 
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
Extreme  Programming    Talk Wise  Consulting   Www.TalkwiseconsultingExtreme  Programming    Talk Wise  Consulting   Www.Talkwiseconsulting
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
talkwiseone
 
Extreme programming talk wise consulting - www.talkwiseconsulting
Extreme programming   talk wise consulting - www.talkwiseconsultingExtreme programming   talk wise consulting - www.talkwiseconsulting
Extreme programming talk wise consulting - www.talkwiseconsulting
talkwiseone
 

Ähnlich wie Cross functional peer review preso 10-01-2013 (20)

Ben Walters - Creating Customer Value With Agile Testing - EuroSTAR 2011
Ben Walters - Creating Customer Value With Agile Testing - EuroSTAR 2011Ben Walters - Creating Customer Value With Agile Testing - EuroSTAR 2011
Ben Walters - Creating Customer Value With Agile Testing - EuroSTAR 2011
 
Software testing
Software testingSoftware testing
Software testing
 
Session on evaluation of DevSecOps
Session on evaluation of DevSecOpsSession on evaluation of DevSecOps
Session on evaluation of DevSecOps
 
ALM with TFS: From the Drawing Board to the Cloud
ALM with TFS: From the Drawing Board to the CloudALM with TFS: From the Drawing Board to the Cloud
ALM with TFS: From the Drawing Board to the Cloud
 
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
Curiosity and Xray present - In sprint testing: Aligning tests and teams to r...
 
Critical Capabilities to Shifting Left the Right Way
Critical Capabilities to Shifting Left the Right WayCritical Capabilities to Shifting Left the Right Way
Critical Capabilities to Shifting Left the Right Way
 
DevOps Presentation.pptx
DevOps Presentation.pptxDevOps Presentation.pptx
DevOps Presentation.pptx
 
Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster Model-Based Testing for ALM Octane: Better tests, built faster
Model-Based Testing for ALM Octane: Better tests, built faster
 
Quantifying DevOps Adoption Empirically for Demonstrable ROI
Quantifying DevOps Adoption Empirically for Demonstrable ROIQuantifying DevOps Adoption Empirically for Demonstrable ROI
Quantifying DevOps Adoption Empirically for Demonstrable ROI
 
Turning Developers into Testers
Turning Developers into TestersTurning Developers into Testers
Turning Developers into Testers
 
SplunkLive! London 2016 Splunk for Devops
SplunkLive! London 2016 Splunk for DevopsSplunkLive! London 2016 Splunk for Devops
SplunkLive! London 2016 Splunk for Devops
 
Agile testing
Agile testingAgile testing
Agile testing
 
Beginners overview of automated testing with Rspec
Beginners overview of automated testing with RspecBeginners overview of automated testing with Rspec
Beginners overview of automated testing with Rspec
 
Application Lifecycle Management
Application Lifecycle ManagementApplication Lifecycle Management
Application Lifecycle Management
 
Resume
ResumeResume
Resume
 
The Magic Of Application Lifecycle Management In Vs Public
The Magic Of Application Lifecycle Management In Vs PublicThe Magic Of Application Lifecycle Management In Vs Public
The Magic Of Application Lifecycle Management In Vs Public
 
Creating a successful continuous testing environment by Eran Kinsbruner
Creating a successful continuous testing environment by Eran KinsbrunerCreating a successful continuous testing environment by Eran Kinsbruner
Creating a successful continuous testing environment by Eran Kinsbruner
 
A Lap Around Visual Studio 2010
A Lap Around Visual Studio 2010A Lap Around Visual Studio 2010
A Lap Around Visual Studio 2010
 
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
Extreme  Programming    Talk Wise  Consulting   Www.TalkwiseconsultingExtreme  Programming    Talk Wise  Consulting   Www.Talkwiseconsulting
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
 
Extreme programming talk wise consulting - www.talkwiseconsulting
Extreme programming   talk wise consulting - www.talkwiseconsultingExtreme programming   talk wise consulting - www.talkwiseconsulting
Extreme programming talk wise consulting - www.talkwiseconsulting
 

Mehr von SmartBear

Mehr von SmartBear (20)

Enforcing Your Organization's API Design Standards with SwaggerHub
Enforcing Your Organization's API Design Standards with SwaggerHubEnforcing Your Organization's API Design Standards with SwaggerHub
Enforcing Your Organization's API Design Standards with SwaggerHub
 
Introducing OpenAPI Version 3.1
Introducing OpenAPI Version 3.1Introducing OpenAPI Version 3.1
Introducing OpenAPI Version 3.1
 
IATA Open Air: How API Standardization Enables Innovation in the Airline Indu...
IATA Open Air: How API Standardization Enables Innovation in the Airline Indu...IATA Open Air: How API Standardization Enables Innovation in the Airline Indu...
IATA Open Air: How API Standardization Enables Innovation in the Airline Indu...
 
The State of API 2020 Webinar – Exploring Trends, Tools & Takeaways to Drive ...
The State of API 2020 Webinar – Exploring Trends, Tools & Takeaways to Drive ...The State of API 2020 Webinar – Exploring Trends, Tools & Takeaways to Drive ...
The State of API 2020 Webinar – Exploring Trends, Tools & Takeaways to Drive ...
 
How LISI Automotive Accelerated Application Delivery with SwaggerHub
How LISI Automotive Accelerated Application Delivery with SwaggerHubHow LISI Automotive Accelerated Application Delivery with SwaggerHub
How LISI Automotive Accelerated Application Delivery with SwaggerHub
 
Standardising APIs: Powering the Platform Economy in Financial Services
Standardising APIs: Powering the Platform Economy in Financial ServicesStandardising APIs: Powering the Platform Economy in Financial Services
Standardising APIs: Powering the Platform Economy in Financial Services
 
Getting Started with API Standardization in SwaggerHub
Getting Started with API Standardization in SwaggerHubGetting Started with API Standardization in SwaggerHub
Getting Started with API Standardization in SwaggerHub
 
Adopting a Design-First Approach to API Development with SwaggerHub
Adopting a Design-First Approach to API Development with SwaggerHubAdopting a Design-First Approach to API Development with SwaggerHub
Adopting a Design-First Approach to API Development with SwaggerHub
 
Standardizing APIs Across Your Organization with Swagger and OAS | A SmartBea...
Standardizing APIs Across Your Organization with Swagger and OAS | A SmartBea...Standardizing APIs Across Your Organization with Swagger and OAS | A SmartBea...
Standardizing APIs Across Your Organization with Swagger and OAS | A SmartBea...
 
Effective API Lifecycle Management
Effective API Lifecycle Management Effective API Lifecycle Management
Effective API Lifecycle Management
 
The API Lifecycle Series: Exploring Design-First and Code-First Approaches to...
The API Lifecycle Series: Exploring Design-First and Code-First Approaches to...The API Lifecycle Series: Exploring Design-First and Code-First Approaches to...
The API Lifecycle Series: Exploring Design-First and Code-First Approaches to...
 
The API Lifecycle Series: Evolving API Development and Testing from Open Sour...
The API Lifecycle Series: Evolving API Development and Testing from Open Sour...The API Lifecycle Series: Evolving API Development and Testing from Open Sour...
The API Lifecycle Series: Evolving API Development and Testing from Open Sour...
 
Artificial intelligence for faster and smarter software testing - Galway Mee...
Artificial intelligence for faster and smarter software testing  - Galway Mee...Artificial intelligence for faster and smarter software testing  - Galway Mee...
Artificial intelligence for faster and smarter software testing - Galway Mee...
 
Successfully Implementing BDD in an Agile World
Successfully Implementing BDD in an Agile WorldSuccessfully Implementing BDD in an Agile World
Successfully Implementing BDD in an Agile World
 
The Best Kept Secrets of Code Review | SmartBear Webinar
The Best Kept Secrets of Code Review | SmartBear WebinarThe Best Kept Secrets of Code Review | SmartBear Webinar
The Best Kept Secrets of Code Review | SmartBear Webinar
 
How Capital One Scaled API Design to Deliver New Products Faster
How Capital One Scaled API Design to Deliver New Products FasterHow Capital One Scaled API Design to Deliver New Products Faster
How Capital One Scaled API Design to Deliver New Products Faster
 
Testing Without a GUI Using TestComplete
 Testing Without a GUI Using TestComplete Testing Without a GUI Using TestComplete
Testing Without a GUI Using TestComplete
 
Hidden Treasure - TestComplete Script Extensions
Hidden Treasure - TestComplete Script ExtensionsHidden Treasure - TestComplete Script Extensions
Hidden Treasure - TestComplete Script Extensions
 
How Bdd Can Save Agile
 How Bdd Can Save Agile How Bdd Can Save Agile
How Bdd Can Save Agile
 
API Automation and TDD to Implement Master Data Survivorship Rules
API Automation and TDD to Implement Master Data Survivorship RulesAPI Automation and TDD to Implement Master Data Survivorship Rules
API Automation and TDD to Implement Master Data Survivorship Rules
 

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 

Cross functional peer review preso 10-01-2013

Hinweis der Redaktion

  1. So one of the key findings of the study is that as you do code review, it is effective in discovering defects until about the 60 minutes mark, you see the defects discovered continued to go up along with the time. But after that mark, it starts to plateau, further time spent do not result in more defects discovered.
  2. As you can see from this set of data, the defects density, in another word, how many defects discovered per thousand lines of code tends to cluster between 0 and 500 lines of code examined per hour. So if you go way too fast, let’s say you looked at 1200 lines of code per hour, the number of defects that you discovered goes way down. So like I tell my daughter who just started to drive, go slow.
  3. This is similar to the last one, don’t pile on thousands and thousands of lines of code for someone to review, they will not be able to focus and detect the errors due to the volume. So do just enough code and have it reviewed, in the 200 to 400 range, which give you the best yield.