Allstate Insurance had a problem. While thoroughly testing each of their more than thirty business systems, they were still failing to provide good service to their clients, agents, and internal customers. The reason was simple. Implementing end-to-end business processes requires more than just running data through a set of separate systems. While focusing on automating unit, integration, and system testing, they had failed to consider the need for system-to-system integration tests―tests that would verify that their business systems passed data correctly, met interface expectations, and synchronized properly. Monika Mehrotra and Sandra Alequin describe how Allstate, with the assistance of Infosys, supplemented their existing test suites with a set of end-to-end tests that provided deeper test coverage, demonstrating proper system operation from beginning to end. In addition, Allstate implemented a test environment that more closely resembled their production environment, discovering defects that had previously escaped into daily operation. Learn the importance of end-to-end, not just piecemeal testing.

1. - 1 -
Automating End To End
Scenarios
Presenters:
Sandra Alequin (Senior Manager, Allstate Insurance)
Monika Mehrotra (Senior Manager, lnfosys Ltd)

2. - 2 -
Enterprise testing (ETest) strategy was established in pursuit of a
strategic objective: to improve agency and customer experience with
a goal of minimizing production problems and reducing the risk due
to major application releases on critical business functions.
ETest strategy implementation included establishing new processes,
tools, team and environment. ETest consists of 200+ applications
with over 600+ backend services and equally high number of
interacting databases.
33 key business functions, which are highly integrated, are tested
end-to-end prior to being put in production under this strategy.
Enterprise Testing (ETest) Strategy at Allstate

3. - 3 -
Executive Summary
Java
Applications
.NET
Applications
Siebel
Applications
Mobile
Applications
Open Web
Applications
Mainframe
Presentation Layer Middle-Tier Data Layer
Mainframe
Web
Servers
Application
Servers
LDAP
MQ
TIBCO
SOAP
REST
JMS
EJBs
JDBC
Store
Procedure
FTP
Mobile
Web
Browsers
Applications
Enterprise Testing Environment – An Overview
DatabaseDatabaseDatabase

4. - 4 -
 Over 200+ application from a mix of legacy (AS400, Mainframes) and
new technologies (.NET, Java)
600+ services on different technologies like SOAP, REST etc
Unresponsive Backend Services can cause failures in tests and hence
need constant pinging to keep them alive
Time consuming Defect Root Cause Analysis due to large number of
systems and integrations
Deactivation of Test IDs post Data Refresh activities
Complexities in Enterprise Testing Environment

5. - 5 -
Validation points introduced across different layers for deeper and wider test coverage
5
Executive Summary
Java
Applications
.NET
Applications
Siebel
Applications
Mobile
Applications
Open Web
Applications
Mainframe
Database
Presentation Layer Middle-Tier Data Layer
Mainframe
Web
Servers
Application
Servers
LDAP
MQ
TIBCO
SOAP
REST
JMS
EJBs
JDBC
Store
Procedure
FTP
Mobile
Web
Browsers
Applications
Existing validation points Newly introduced validation points
Solution to address E-Test Environment challenges
DatabaseDatabase

6. - 6 -
Executive Summary
• Implementation of trouble
shooting at service layer to
reduce effort spent root
cause analysis
• Reduction in time spent on
debugging and enable faster
fixes for defects
• Validation of middle tier
components that otherwise are
not tested as part of GUI testing
scenarios
• Deeper and wider coverage
introduced into architecture
with middle tier validation
• Validated integration and
interaction points ensuring
quality in systems
integration
• Data navigating from one
end to another in network is
tracked for quality and
reliability
Smoother Integration Better Coverage Defect Isolationa b c
Users - Presentation layer
Headless clients
consuming services
Data Access Logic Data Utilities Data Sources
Presentation – Middle Tier Integration
Middle Tier – Data Layer Integration
Service Service Service Service Service Service Service
b
a
a
c
c a
b
Addition of Middle Tier Validation in E-Test

7. - 7 -
Complexity, size and time to execute E-Test regression suite
warranted the need to automate the regression suite
An assessment on then E-Test processes were performed and an
Integrated (frontend and backend) automation solution was arrived
at to target improvement in efficiency and effectiveness of E-Test
regression suite execution by giving deeper and wider test coverage.
De-risked and cost effective 2-Phased approach was adopted to build
out the integrated automation.
Phase 1 was performed for feasibility analysis and devise solution
for automation requirements.
Phase 2 was implementing the solution and rollout
Necessity for Integrated Test Automation Solution

8. - 8 -
QTP Automation
Framework
Service Validation
Framework
Frontend Layer
UI Components
Presentation Layer Logic
Backend Service Layer
Service Orchestration
Services
Backend Data Layer
Data Access Components
Data utilities
Data Validation
using CA LISA
E Test Environment
Leverage LISA for
early validation at
service/data layer
Validating UI layer
on a more stable
Backend
Integrated Automation Solution
HP Quality
Center
HP Quick Test
Professional
CA LISA
CA LISA
Backend (Service and Data layer) validation will be automated with CA LISA and Frontend (UI
layer) validation will be automated with HP QTP. The solution will be orchestrated using HP QC
and provides flexibility to validate layers in integrated or silo mode.

9. - 9 -
Switch of test machine is
required to continue execution
Stop for Batch Cycle
Resume after Batch Cycle
Data needs to be captured
and transferred between
the test scripts/test sets
LISA Scripts integration with
QTP and HPQC for backend
scripts execution
Typical ETest End-to-End Scenario for Automation
? – Questions to be answered and solutions to be proven out
Customer does inquiry and requests for a
Quote
Mobile Search Web Search
Agent contacts customer and the quote is converted to a request for
new business
New business is received and reviewed by underwriters
Home Office Staff verifications Underwriters
Underwriter provides approved , New business documents generated and Customer access documents
Document Management
System
Customer Access
Endorsement
confirmation
Policy EndorsementRegistration
Renewal Validations
Validate Renewals processing after payments
Policy Endorsements, Renewals and billing
Endorsement
Documents
Renewal of
Policies
Renewal
Billing
Renewal
Documents:
Endorsement
processing
Renewals Payments
Sample ETest Scenario
?
?
?
?
Exclusive Agent
Applications
Independent Agent
Applications Home Office Staff

10. - 10 -
 Tests were run on specific workstations to mimic customer and user
experience. So how to switch test machines to continue automated
execution?
 How to handle the execution of a single test case across multiple
job batches?
 How would the data required for execution across batch of same
test case maintained throughout the execution of the case, which
may span across days?
 How to integrate HPQC (Test Management), HPQTP (UI Automation)
and CA LISA (Service layer validation) tools to perform an integrated
test run?
Automation Challenges in ETest End-to-End Scenarios

11. - 11 -
Test Automation challenges addressed
Data Extraction
& Persistence
Multiple Machines
Execution
Execution Inline
with Daily Batch
cycles
LISA Scripts integration
with QTP and HPQC
1 2 3 4
Customer does inquiry and requests for a
Quote
Mobile Search Web Search
Agent contacts customer and the quote is converted to a request for
new business
New business is received and reviewed by underwriters
Home Office Staff verifications Underwriters
Underwriter provides approved , New business documents generated and Customer access documents
Document Management
System
Customer Access
Endorsement
confirmation
Policy EndorsementRegistration
Renewal Validations
Validate Renewals processing after payments
Policy Endorsements, Renewals and billing
Endorsement
Documents
Renewal of
Policies
Renewal
Billing
Renewal
Documents:
Endorsement
processing
Renewals Payments
Sample ETest Scenario
2
1
3
4
Exclusive Agent
Applications
Independent Agent
Applications Home Office Staff

12. - 12 -
12
# 1-Data Extraction & Persistence
Data Persistence in XML Files
DataPersistenceacross
machines
DataPersistence
acrossbatchruns
Customer Inquiry, Quote & New Business
Customer Machine:
Customer Inquiry and
Begins Property Quote:
(Mobile and Web)
Agent Machine:
New Business
Purchase
Under Writer
Machine; New
Business
Agent Machine:
Endorsement : Agent
Customer Machine:
Customer: Login to
“Customer Account",
Register
Batch 1: Customer
Inquiry and Begins
Property Quote: (Web)
Batch 2: New Business
Purchase: Agent
Batch 3: New Business :
Agent
Data is getting read by
different machines from XML
Reference # is persisted in XML and used by script for Policy # generation
1
1
Policy # is persisted used by script and later used other scripts2
2
1 2
•Data is persisted in Shared LAN drive which is accessible by all machines
•Data is persisted in form of XMLs for each test case.
•XPATH Queries are employed for faster data retrieval

13. - 13 -
13
Test Execution Dependency- Across Batches
Customer Inquiry, Quote & New Business
Customer Agent Search :
(Mobile and Web)
Customer creates Quote
Customer Inquiry, Quote & New Business
Manage Agency by
Agent
Purchases quote and
email validation
Validation of New
Business
Batch 2
• Automated Script generates Execution Control XML for each Test Case
• Test Script execution status with other details are captured in XML structure
• Execution sets Test Set_Status “Y” in case of Pass “N” for failure
• Next Test Script reads XML data and determines if execution is required or not
Invoke Next
Batch test case if
Status=“Y”
Stop Execution
for test case if
Status=“N”
STOP
Determine Execution
is required/not from
XML
Batch 1
# 2 - Execution Inline with Daily Batch cycles
Cross Test Set Dependency with XMLs

14. - 14 -
14
1A
1B
1C
E Test Release 1
HP QC Test Lab Folders
at Scenario level
HP QC Test Lab Folder at Release level
Batch1
Batch2
BatchN…
Batch1
Batch2
BatchN…
Batch1
Batch2
BatchN…
HP QC Test Sets at
Batch run levels
Each Test Set contains specific test scripts to be run as part of E Test Case
HPQC identifies available machine in the host group and triggers
execution on the available machine within the group. Upon execution
completion, test results are reported back to quality center and execution
is moved to next test script
Machine Groups
HP QC Test Cases
Batch 1 for 1A contains scripts to be
executed before batch cycle process
runs
#3 - Multiple Machines Execution
Hosting in HPQC

15. - 15 -
15
# 4 - CA LISA integration with HPQTP & HPQC
CA LISA

16. - 16 -
16
HP Quality
Center
Mainframe Web Java
Enterprise Automation Components
(Reusable Functions, Keyword functions, Framework Driver Engine)
Platforms Under Test
Automation Framework
Remote Machines
Testers
Java Specific
Components
Web Specific
Components
Mainframe Specific
Components
Framework to handle Application Diversity

17. - 17 -
Test Case Modeling
17
Application
Workflow
Payment
Work
flow
Data
Data1
Data
Work
flow
Work
flow
Data
Data2
Data3
Data1
Data2
Data3
Data1
Data2
Data3
Modularizing the application workflow
by creating multiple reusable Business
Process Components (BPCs)
Workflow created in
excel using action
keywords
Multiple data sets can be mapped to
the same workflow to easily create
different data oriented test cases
Customer
Registration
Agent Search
Diagrammatic representation of test case modeling
Test Case modeling with the framework

18. - 18 -
Test Case Modeling
18
ETest Implementation Approach
Assessment of ETest
Environment and Test
Scenarios
ETest Automation
Feasibility Analysis
ETest Implementation
Design
ETest SOA Validation
Implementation
ETest SOA validation
rollout
ETest UI Automation
Feasibility Analysis
ETest Automation
Unique requirements
identified
ETest Automation
Solution Development
for requirements
Proof Of Concept for
ETest Automation
Complete Automation
Implementation in
ETest
Rollout of Automation
Testing for ETest

19. - 19 -
Executive Summary
Current State Post Implementation Benefits
Quality
• Test steps scoped
based on
available effort
• End to End
Business scenario
validated in base
state
• Volume test steps
execution was increased
by 3 times
• End to End Business
scenario implemented
in multiple states
• 200% improvement across Tests and
ROCs
• Improved coverage at backend layer
• Increased and consistent validation
• Multiple Regression Runs in current
window
Time
• 3-week window
to execute E2E
tests
• 1-week window to
execute E2E tests
• 33% reduction in the execution
window
• Creating window for more execution
• Reduced turn around time on defect
resolution
Cost
• High manual
effort to rerun
failed scenarios
• Overall team is
reduced by 40%
• 30% reduction in execution cost
• Early validation
• Reduced issues and effort during End
to End
• Lessen manual effort with automation
ETest Automation - Post Implementation Analysis

20. - 20 -
Executive Summary
Additional Benefits of Integrated Automation Solution
Predictability
 Validation at backend layer resulting in early defect detection
 Post implementation UI execution happens on a validated backend
layer
Portability
 Solution can be deployed on E-Test or other environments with
minimal effort
Coverage
 Improve coverage by introducing more validations at all layers of
validation
Modularity &
Scalability
 Deploy solution in packages to enable selective run
 Ability to run automation scripts for specific scenario category
and/or applications.

21. - 21 -
Questions & Answers

22. - 22 -
Thank you.