SlideShare ist ein Scribd-Unternehmen logo
1 von 12
Downloaden Sie, um offline zu lesen
TK
PM Tutorial
10/1/2013 1:00:00 PM

"Production Performance
Testing in the Cloud"
Presented by:
Dan Bartow
SOASTA

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Dan Bartow
SOASTA, Inc.
At SOASTA, Dan Bartow is Vice President of Product Management for the industry leading and
award-winning product CloudTest. Prior to joining SOASTA, Dan was Senior Manager of
Engineering at Intuit where his team was responsible for the speed and stability of TurboTax
Online. During the past decade he has been responsible for the performance of websites for
dozens of leading consumer brands.
8/27/2013

Create a One-Page Capacity
Model for High-Traffic Web
Applications
Dan Bartow
SOASTA

About the Speaker
SOASTA
VP Product Management
CloudTest Evangelist

Intuit
Sr. Manger, Engineering
TurboTax Online and E-com

ATG
Sr. Deployment Engineer

Work
American Airlines, Best Buy, Target, Turbotax Online, QuickenOnline, MySpace,
Dennys, Dominos, Mattel, Hallmark, FAA, US Army, AT&T Wireless, Alcatel,
Newsweek, Oprah, NeimanMarcus, SBC, Plantronics, Kodak, JCrew, Cingular,
Newell Rubbermaid

1
8/27/2013

Poor performance.

The Problems
1.

No knowledge of what the critical metrics are at each tier
of an applications architecture

1.

No record of what the critical metric values were at the
peak last year

1.

No idea of what those metrics should be at the next peak
traffic day

1.

No indication of what the capacity is today and how that
relates to the upcoming peak

2
8/27/2013

The Solution
(or at least a huge step in the right direction)

The one page capacity model

What one looks like

3
8/27/2013

How to build it

Step 1.
Create a simple architecture diagram, containing mostly
infrastructure, that lists key technologies in play for each tier

You don’t need this

4
8/27/2013

Getting closer

What to include in the diagram

1.
2.
3.

Key infrastructure components
Critical services and their infrastructure where possible
Third party components (if applicable)

Hint: If you can’t test it, you probably don’t need to include it. #controversial

5
8/27/2013

A reasonable diagram

How to build it

Step 2.
Identify the metrics that should be tracked for each tier

6
8/27/2013

Three layers of monitoring
Layer 1 - Customer

Concurrent sessions
Pages (requests) / sec

Layer 2 - Server

CPU %
Memory
Network IO
Disk IO

Layer 3 - Container

JVM Heap usage
Threads / second
Login servlet invocations / sec
Database connections

To make this all happen (and be
really useful)

You need comprehensive monitoring
You need to be able able to test to scale

7
8/27/2013

Metrics examples for common
technologies

Metric examples for common
technologies
CPU
Memory
TX/sec (especially SSL TX)
Bandwidth/sec
CPU
Memory
Threads / sec

Container

Application

CPU
Memory
Threads or processes / sec

Key GET/POST actions
Key method calls
External connections

Container

Application

CPU
Memory
JVM heap usage
Garbage collection interval
Threads / sec

Key servlet invocation count (login, order
placement)
Database connection count
Key servlet execution time

Container
CPU
Memory
Connections
Queries / sec
(SELECT/INSERT/UPDATE/
DELETE)
Table or row locks

8
8/27/2013

How to build it

Step 3.
Fill in the values for your peak last year (the peak second)

How to build it

Step 4.
Project and do simple math for the upcoming year. Base
this math on top level percentages and honor relationships
between metrics where needed.

9
8/27/2013

Contact Information
Dan Bartow
VP Product Management
SOASTA Inc.

Email:
Twitter:
LinkedIn:
Blog:

dan@soasta.com
www.twitter.com/PerfDan
www.linkedin.com/in/danbartow/
www.soasta.com

10

Weitere ähnliche Inhalte

Mehr von TechWell

Mehr von TechWell (20)

System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 
Scale: The Most Hyped Term in Agile Development Today
Scale: The Most Hyped Term in Agile Development TodayScale: The Most Hyped Term in Agile Development Today
Scale: The Most Hyped Term in Agile Development Today
 
Measure DevOps for Objective Continuous Improvement Practices
Measure DevOps for Objective Continuous Improvement PracticesMeasure DevOps for Objective Continuous Improvement Practices
Measure DevOps for Objective Continuous Improvement Practices
 
Microservices and Docker at Scale: The PB&J of Modern Systems
Microservices and Docker at Scale: The PB&J of Modern SystemsMicroservices and Docker at Scale: The PB&J of Modern Systems
Microservices and Docker at Scale: The PB&J of Modern Systems
 

Kürzlich hochgeladen

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Kürzlich hochgeladen (20)

Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 

Production Performance Testing in the Cloud

  • 1. TK PM Tutorial 10/1/2013 1:00:00 PM "Production Performance Testing in the Cloud" Presented by: Dan Bartow SOASTA Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Dan Bartow SOASTA, Inc. At SOASTA, Dan Bartow is Vice President of Product Management for the industry leading and award-winning product CloudTest. Prior to joining SOASTA, Dan was Senior Manager of Engineering at Intuit where his team was responsible for the speed and stability of TurboTax Online. During the past decade he has been responsible for the performance of websites for dozens of leading consumer brands.
  • 3. 8/27/2013 Create a One-Page Capacity Model for High-Traffic Web Applications Dan Bartow SOASTA About the Speaker SOASTA VP Product Management CloudTest Evangelist Intuit Sr. Manger, Engineering TurboTax Online and E-com ATG Sr. Deployment Engineer Work American Airlines, Best Buy, Target, Turbotax Online, QuickenOnline, MySpace, Dennys, Dominos, Mattel, Hallmark, FAA, US Army, AT&T Wireless, Alcatel, Newsweek, Oprah, NeimanMarcus, SBC, Plantronics, Kodak, JCrew, Cingular, Newell Rubbermaid 1
  • 4. 8/27/2013 Poor performance. The Problems 1. No knowledge of what the critical metrics are at each tier of an applications architecture 1. No record of what the critical metric values were at the peak last year 1. No idea of what those metrics should be at the next peak traffic day 1. No indication of what the capacity is today and how that relates to the upcoming peak 2
  • 5. 8/27/2013 The Solution (or at least a huge step in the right direction) The one page capacity model What one looks like 3
  • 6. 8/27/2013 How to build it Step 1. Create a simple architecture diagram, containing mostly infrastructure, that lists key technologies in play for each tier You don’t need this 4
  • 7. 8/27/2013 Getting closer What to include in the diagram 1. 2. 3. Key infrastructure components Critical services and their infrastructure where possible Third party components (if applicable) Hint: If you can’t test it, you probably don’t need to include it. #controversial 5
  • 8. 8/27/2013 A reasonable diagram How to build it Step 2. Identify the metrics that should be tracked for each tier 6
  • 9. 8/27/2013 Three layers of monitoring Layer 1 - Customer Concurrent sessions Pages (requests) / sec Layer 2 - Server CPU % Memory Network IO Disk IO Layer 3 - Container JVM Heap usage Threads / second Login servlet invocations / sec Database connections To make this all happen (and be really useful) You need comprehensive monitoring You need to be able able to test to scale 7
  • 10. 8/27/2013 Metrics examples for common technologies Metric examples for common technologies CPU Memory TX/sec (especially SSL TX) Bandwidth/sec CPU Memory Threads / sec Container Application CPU Memory Threads or processes / sec Key GET/POST actions Key method calls External connections Container Application CPU Memory JVM heap usage Garbage collection interval Threads / sec Key servlet invocation count (login, order placement) Database connection count Key servlet execution time Container CPU Memory Connections Queries / sec (SELECT/INSERT/UPDATE/ DELETE) Table or row locks 8
  • 11. 8/27/2013 How to build it Step 3. Fill in the values for your peak last year (the peak second) How to build it Step 4. Project and do simple math for the upcoming year. Base this math on top level percentages and honor relationships between metrics where needed. 9
  • 12. 8/27/2013 Contact Information Dan Bartow VP Product Management SOASTA Inc. Email: Twitter: LinkedIn: Blog: dan@soasta.com www.twitter.com/PerfDan www.linkedin.com/in/danbartow/ www.soasta.com 10