This document discusses Mansfield Energy Company's journey to enhance its QA strategy to achieve agile quality engineering. It covers establishing the right test strategy, ensuring the QA team has the proper skills and tools, and aligning the quality assurance process with an agile landscape. Key points include building an agile test strategy, anticipating changes, establishing a quality assurance center of excellence, implementing continuous testing, and measuring success through analytics and accountability. The overall goal is to deliver high quality software through an improved QA governance framework.

1. ©1996-2019 Aspire Systems, Inc.
Enhancing Your QA
Strategy to Achieve
Agile Quality
Engineering:
Our Journey
2019

2. ©1996-2019 Aspire Systems, Inc.
2
Speakers
Vasanth Manickam
Technical Test Manager,
Aspire Systems
Katie Kavanaugh
QA Director,
Mansfield

3. ©1996-2019 Aspire Systems, Inc.
3
Quality Maturity Level
About Mansfield
Welcome to Mansfield Energy

4. ©1996-2019 Aspire Systems, Inc.
4
A Great Place to Work - 10
Glorious Wins in a Row
Quality Maturity LevelTechnology Awards
Forrester Rated Aspire
one of the Top
Continuous Testing
Service Providers (2017)
About Aspire
Global technology services firm with
core DNA of Software Engineering
Specific areas of expertise around Software
Engineering, Digital Services, Testing and
Infrastructure & Application Support
International headquarters in
Singapore with presence across US,
UK, Benelux, Middle East and India
Recognized 10 consecutive times
as “Best Place to Work for” by
GPW Institute
Vertical focus among Independent
Software Vendors and Retail,
Distribution & Consumer Products
3000+ employees; 150+
active customers
CMMI Maturity Level 3, ISO
9001:2015 and ISO 27001 : 2013
certified

5. ©1996-2019 Aspire Systems, Inc.
5
Middle East
India
Singapore
Poland
NetherlandsUK
US
Mexico
Global Presence

6. ©1996-2019 Aspire Systems, Inc.
6
Preface
1 Before you Start
2 Organizational Strategy and Goals
3 The Right Test Strategy
4 Landscape Alignment and Futuristic Relevancy
 Knowing your Company’s Business
 Do you have the Right Team?
 People, Tools and Structure
 Agile Governance, Policy and Process Model
 Static vs Dynamic
 Anticipating Change
 Landscape Alignment
 Measuring for Success

7. ©1996-2019 Aspire Systems, Inc.
7
Before You Start

8. ©1996-2019 Aspire Systems, Inc.
8
Knowing your Company’s Business
Before you strategize make sure you are in the know…
Do you
understand both
long and short
term tactical and
strategic
objectives?
What systems
will you
support?
What is the
expectation of
your
organization?
What are you
solving for your
company?
Who is your
company and
what is it’s
business?
Mansfield Energy Company and the Industry

9. ©1996-2019 Aspire Systems, Inc.
9
Having the Right Team
Are you building a new team, adding to existing, or retaining?
Does the current team fit the skills required to deliver the products at or above expected
Quality SLA requirements?
Team realignment is a required success factor and is not a one time event.
Team considerations:
 Seasoned vs. Beginner: Both serve a different
purpose and provide excellent value
 Skills Platform: Manual vs Automation
 Is Domain knowledge critical to success?
 Onshore, Offshore, Blended
 Front and Backend skills required
 Who are your Leads?
Critical Skills: Complexity of the system
– Having UI/Design skills does not equate
to Accounting/Complex Algorithm skills!
Product Mapping
Prakash
Amit
Sudhindra
Ashwini
Biradar
Lakshmi
Rohit
Ent ANA User Administration B E I B I E E Legend
Ent ANA Data Level Permission B E I B I I I Beginner B
Ent ANA Group Administration B E I B I E E Intermediate I
Ent ANA Role Administration B E I B I I E Expert E
Ent ANA Application Module B E I B I I E
Ent ANA Role Module Permission B E I B I I E
Ent ANA Company Administration B E I B I E E
Ent ANA Permission B E I B I E E
Ent ANA Permission Type B E I B I I E
Ent ANA Password Rule Configuration B E I B I E E
Ent Orders Order Delivery Console - DEF I I I I I E
Ent Orders Order Delivery Console - LTL I I I I I E
Ent Orders Order Delivery Console I I I I I E
Ent Orders PO
Ent Orders Route Builder I I E
Ent Orders Sales Order I I I I I E
Ent Orders Schedule Deliveries I B I I I E
Ent MDM Configuration B I I B I I
Ent MDM Companies I E I B I E
Ent MDM Product B E I I E
Ent MDM Locations I E I I E
Ent MDM Contracts I E I B I E
Ent MDM Governance I E
An Agile program
cannot be
successful without
team unity and
skills alignment.

10. ©1996-2019 Aspire Systems, Inc.
10
Unified Team Commitment
Deliver on Quality by focusing on Excellence -
Provide customer value with the goal of
exceeding customer satisfaction in the delivery of
our products and services by implementing
Mansfield best practices, driving team
development & recognition, and by empowering
our employees. Developing a culture that’s
passionate about quality and accountability.
Business
Technology
Quality
Statement…
Aligning with
our Core
Principles
 Conscientiousness
 Innovation
 Integrity
Mansfield Core Principles:
 Personal Service
 Excellence

11. ©1996-2019 Aspire Systems, Inc.
11
Organizational
Strategy & Goals

12. ©1996-2019 Aspire Systems, Inc.
12
Quality Strategies: Our Quality Directive
Balancing the triple constraints…
Scope
Balanced Delivery
Time
Commitment to schedule.
Resources
With the right team, tools
and budget.
Scope management as the focal point with appropriate risk based analysis,
planning and mitigation for Resources and Time –
And Quality being the Constant.

13. ©1996-2019 Aspire Systems, Inc.
13
Design Verification
Usability Assessment (Ease of Use)
Requirements Validation and Traceability
Bug finding mission
Feature Acceptance
Environmental and External Systems alignment
Look and Feel
Performance Assessment
Security and Intrusion
Goal of Validation & Verification
Strategic: To establish stakeholder confidence in the system.
Tactical: Varies project by project. The tactical testing goal(s) and objectives definition occurs during the Plan phase.
What activities must QA perform in order to prove system meets criteria
established by the business in order accept the product delivery?

14. ©1996-2019 Aspire Systems, Inc.
14
BT: QA* Governance Framework Goals
Better manage financial footprint of quality by pulling
quality curve in and reducing the “Cost of a Defect”.
Deliver High Quality Software & Services.
Provide Greater Transparency in Risks and
Dependencies to the Business.
Provide Real Time and Proactive Guidance
on Quality Trends and Analytics.
Drive Adherence to Compliance for Quality,
Security & Fraud, and Data.
*QA being defined as true end to end program quality assurance from
project Initiation to Production side implementation.

15. ©1996-2019 Aspire Systems, Inc.
15
Program Quality Assurance Center of Excellence
 Business alignment
 Compliance
 Guidance
 Accountability
 Security & Fraud
 Policy & Process
 Tools and Services
 Base/Functional/Reg
 Performance & Load
 CI/Security/Dev Tools
 Dedicated Team
 Status – Prog/Proj
 Quality Accountability
 Risk Management
 Training & Mentorship
 Compliance
Measuring for Success
 Enhanced Services
 Test Plan & Execution
 Dedicated Team
 Artifact & Issue Management
 UAT Framework
QA Testing as a Service
Governance
Automation &
Tools COE

16. ©1996-2019 Aspire Systems, Inc.
16
Quality Benefits from Governance Alignment
 Reduction in Customer
side defects
 Lower defect escape
ratios per product cycle
 Shorter and tighter
quality curve
 Overall reduction of
the cost of a defect
 Higher quality product
deliverables
 Increase in Customer
satisfaction
 Consistent and
repeatable quality
Leading to

17. ©1996-2019 Aspire Systems, Inc.
17
The Right Test
Strategy

18. ©1996-2019 Aspire Systems, Inc.
18
Building the Agile Strategy
Key to building an Agile Strategy: Remove the policy and make it standard behavior!
Static = Policy Dynamic = Strategy
 SDLC Runbook
 QA Testing Goals & Acceptance
 Coverage Approach
 Entry and Exit Criteria
 Change & Defect Management
 Test Deliverables & Metrics
 Environments, Data, Tools
 UAT Framework
 Severity and Priority Classification
 Root Cause Analysis
 Unit Test and Code Review Practices
 Definition of Done
 Warranty Period
 Introduction
 Scope
 Feature Validation
 Approach & Exceptions to Policy
 Constraints/Risk/Mitigation
NOTE: QA Process and Policy is
methodology agnostic. Process
‘snaps-in’ to the model being
used, whether Agile, Waterfall,
Blend or other

19. ©1996-2019 Aspire Systems, Inc.
19
Agile Strategy Approach
Embed Tactical Testing as an
Integral part of the software
lifecycle
 Program Strategy should focus
on the Policy
 Validation Strategy should focus
on the Project (body of change)
Schedule, Scope, Resources are flexible but
Quality must remain constant.
 Continuous feedback at each stage of the
software delivery cycle
 Consider different avenues of validation to
leverage a blended approach
Clearly define testing
approaches, relevancy
and timing

20. ©1996-2019 Aspire Systems, Inc.
20
Anticipate Change and Challenges
What is your checklist before you begin:
Our System is foundationally complex and
layered
Transformation from a legacy system setup
 Pricing
 Product
 Taxes
Requires frequent releases – Crunched
timelines for Testing
Constant change adds overhead to test
automation lab management
Frequent application changes in agile
environment - High maintenance
 Extras/Customizable
 Seasonal
 Delivery
Distributed and complex applications

21. ©1996-2019 Aspire Systems, Inc.
21
Agile Planning Tricks
 Shift Left: Don’t wait until the QA phase to start validation
 Test driven development: Writes the test scenario before writing
code or even C.C.A.T.s
 Involving UAT/Business users in parallel with the integration test
phase
 Ensure test scenario are written to drive out user acceptance
 Employ continuous performance and system monitoring
 Test environment with right test data
 Incremental test automation
 Automated quality gates - Fail fast
 Identify, automate and execute right set of tests at the
right time
 Risk based/Exploratory testing
 Non - Functional Testing: Performance & Security
 Scalable to future needs - Security testing, business
process automation
 Reduced maintenance effort
 Integrated Solution
 Build, Quality Checks, Monitoring
Selecting right tools,
technology & framework
Selecting right tools,
technology & framework
TestingContinuous Testing
Development‘Shift Left’ vs ‘Shift
Right’ Testing
Iteration 0

22. ©1996-2019 Aspire Systems, Inc.
22
Continuous Testing Approach
Testing over each
stage of development
Performance:
Compare against
baseline
Run for every new
code deployment
Through automated
smoke & regression
Business process
automation
01
02
03
04
05
Target scope areas
Smoke – Non-Transactional: Page
navigation, Menu validation, Search etc.
Core Regression – E2E
Full Regression – Transactional
Performance – Baseline, Load & Stress test

23. ©1996-2019 Aspire Systems, Inc.
23
Test Automation – Schedule Plan
Test Suite Schedule(in IST) Purpose
Functional
Smoke
New deployment Verify the new feature deployments
After every DB Refresh Smoke test after the DB refresh
On request To validate the readiness of new environments
Core & Full Regression Every alternate week
To verify that new feature deployments doesn't break the existing
functionalities
Category Based
Self serve version where the user can choose the test suite based on the
build variables -
* Test Suite(Smoke/Sanity/Regression)
* Module
* Test Environment(QA/Staging)
* Browser(Firefox/Chrome/IE)
Non-Functional
Performance Baseline, Load and
Stress Test
Weekly
To ensure that there is performance degradation due to new feature
deployments
On Request Check the stability after DB refresh

24. ©1996-2019 Aspire Systems, Inc.
24
Mansfield’s Test Automation – Proven Results
Cost effective test automation solution (zero
cost) through open source tools
Less script maintenance as a result of
robust framework
~80% of manual regression test execution cost is
cut down through automated tests
Faster feedback: Bugs were reported in an average
turnaround time of less than 1.5 hours post deployment
Faster time to market
Quality gatekeeper for code promotion to Prod
Improved Product Quality
 Reduced regression bugs in production: 300+ bugs logged
 Manual and automation efforts combined delivered a significant
reduction of Defect Escape ratio from over 30 to a DER of only 3%
 Better performance experience for the users

25. ©1996-2019 Aspire Systems, Inc.
25
Landscape
Alignment &
Futuristic
Relevancy

26. ©1996-2019 Aspire Systems, Inc.
26
Do you have the right tools/infrastructure?
AI Tools
Development
Tools
Code Quality
QA
Automation
Performance
Environments
Monitoring
Test Case
Management
Defect
Management
Configuration:
CI/CD
Security /
Intrusion
Document
Management
& More!
Continuous?
Relevancy?
Futuristic?
Secure?
Reportable?
Integrated/Seamless?
Current Version?
Embracing Technology
and Automation!

27. ©1996-2019 Aspire Systems, Inc.
27
Measuring for Success
Analytics, Reports, and Accountability: Generally, the last governance items to be
implemented but some of the most important!

28. ©1996-2019 Aspire Systems, Inc.
28
Quality & Personal Accountability
The quality of a great product lies in the
hands of the individuals ‘creating
it’…Standards, specifications,
committees, and change controls will not
assure quality. …It is the commitment of
the individuals to excellence, their
mastery of the tools of their crafts, and
their ability to work together that makes
the products not the rules
Cem Kaner J.D., Ph.D.
(Florida Institute of Tech)

29. ©1996-2019 Aspire Systems, Inc.
Q &A

30. ©1996-2019 Aspire Systems, Inc.
Thank You
christina.sridhar@aspiresys.com