Oracle's Application Testing Suite is an integrated, full lifecycle solution that enables customers to define and manage their application testing process, validate application functionality, and ensure that their applications will perform under load. With Application Testing Suite, customers can deploy Web applications and Web Services in less time while maximizing the efficiency of testing team. For Oracle packaged applications, Accelerators are available to provide an out-of-box testing solution designed specifically for E-Business Suite and Siebel applications.

1. Application Performance Management – Virtual Classroom
Mayur Palta
Senior Technical Consultant

2. The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
© 2009 Oracle Corporation – Proprietary and Confidential

3. Agenda
• Application Quality Management
• Application Testing
• Oracle’s Approach to Application Quality
• Test Data Management
• Application User Experience Management
• Managing Real User Experience
• Application Operations Management
• Oracle Enterprise Manager
• Next Steps

4. What is Application Testing?
• A set of activities conducted through scripts with the
intent of finding errors in software.
• An activity being part of the software development
process aimed at evaluating a software item (system,
subsystem, unit etc.) features (functionality,
performance etc.) against the given set of system
requirements.
• Process of verifying that a program functions properly.

5. Business Impact of Application Quality
• Poor software quality puts your investments at risk
• Cost to fix an error found after product release was up to 100 times more
than one identified in the design phase [Source: National Institute of Standards & Technology,
2002]
• About 80 percent of development costs are consumed by software
developers identifying and correcting defects [Source: IBM: The business value of software
quality.]
• Testing typically accounts for 30 to 50% of a software project budget &
non-reproducible or ad-hoc testing is of little or no use [Source: Pullam & Doyle, 1998]
[Source: W.R. Blischke, D.N.P. Murthy. "Reliability", Section 9.5.1, pp. 311.]

6. Common Testing Cycles
Traditional Ways
15% Tested
Application Groups Own Testing
Used Application Testing Tools
Simulation Queries
Very Scripting & Labor Intensive
85% Untested
Problems
Limits the Testing
Not a Production Test
Application Performance Problems Deploying an Untested System
predominately link back to the SQL
Queries in the Database Vulnerable to Poor Performance

7. A Lifecycle Approach to Quality
1. Design Test Plans 2. Develop Manual Test
Based on Application Cases and Automated
Requirements Test Scripts
4. Execute Load Tests 3. Execute Functional
and Tune Application Tests to Validate
Performance Application
Requirements

8. Application Performance Testing & Management
Real User
Experience Insight
Application
Testing Suite
Enterprise
Manager
Application Lifecycle

9. Application Performance Testing & Management
Application
Testing Suite
Application Lifecycle

10. Oracle Testing History
Acquired e-Test Suite from Empirix in June 2008
• 10+ years of automated testing experience
• Currently Working with Internal Application Developers
• Specialist in testing web-based application
• .NET
• JAVA
• Coldfusion

11. Oracle Application Testing Suite
A powerful, integrated testing solution
Functional Testing for Web Apps
Test Manager for Web Applications
Load Testing for Web Apps

12. Oracle Application Testing Suite
A Lifecycle Approach to Quality
Oracle Test Manager for
Web Applications
1. Design Test Plans 2. Develop Manual Test
Based on Application Cases and Automated
Requirements Test Scripts
4. Execute Load Tests 3. Execute Functional
and Tune Application Tests to Validate
Performance Application
Requirements

13. Oracle Test Manager for Web Applications:
Test Process Management
• Manage test process from
centralized Web-based
console
• Define test requirements
• Develop manual &
automated test cases
• Document and track
defects
• Create reports

14. Oracle Application Testing Suite
A Lifecycle Approach to Quality
Oracle Test Manager for
Web Applications
1. Design Test Plans 2. Develop Manual Test
Based on Application Cases and Automated
Requirements Test Scripts
Oracle Functional
Testing for Web
Applications
4. Execute Load Tests 3. Execute Functional
and Tune Application Tests to Validate
Performance Application
Requirements

15. Oracle Functional Testing for Web Applications:
Automated Functional & Regression Testing
• Automate transactions for
Web applications & Web
Services
• Execute rigorous
functional test cases
• Create automated
regression test suites
• Identify and report on
functional application
failures

16. Oracle Application Testing Suite
A Lifecycle Approach to Quality
Oracle Test Manager for
Web Applications
1. Design Test Plans 2. Develop Manual Test
Based on Application Cases and Automated
Requirements Test Scripts
Oracle Load Oracle Functional
Testing for Web Testing for Web
Applications Applications
4. Execute Load Tests 3. Execute Functional
and Tune Application Tests to Validate
Performance Application
Requirements

17. Oracle Load Testing for Web Applications:
Load and Performance Testing and Tuning
• Create realistic load test
scenarios that simulate
end-user behavior
• Scale to thousands of
concurrent users
• Perform functional content
validation under load
• Monitor server-side
performance & correlate to
end-user response times
• Isolate and resolve
performance bottlenecks

18. Need for Data Masking
Test Data Management
• Key Drivers
• Privacy and compliance
LAST_NAME SSN SALARY
• HIPAA, Breach Notification Laws AGUILAR 203-33-3234 40,000
• EU Data Privacy Directive BENSON 323-22-2943 60,000
• Application testing D’SOUZA 989-22-2403 80,000
• Offshore application development FIORANO 093-44-3823 45,000
• Offshore / In-house software QA
• Key Requirements
LAST_NAME SSN SALARY
• Support database and application ANSKEKSL 111—23-1111 40,000
referential integrity
BKJHHEIEDK 111-34-1345 60,000
• Minimal performance impact KDDEHLHESA 111-97-2749 80,000
• Protect against reverse FPENZXIEK 111-49-3849 45,000
transformation

19. Data Masking Pack
Oracle Enterprise Manager
Production
• Automates production data masking
• Mask data from a production database
• Define rules once
• Data relationship discovery
• Automates data relationship
enforcement using existing foreign keys
Clone/
• Enter custom data relationships known Mask Staging
to the application
• Rules repository
• Format library, masking definitions
• Testing
• View sample data before masking
Test Test

20. Application Performance Testing & Management
Real User
Experience Insight
Application
Testing Suite
Enterprise
Manager
Application Lifecycle

21. Challenges
Business Challenge IT Challenge
Line of Business Apps Manager / IT Operations
(LOB) Manager Apps Support
• Low transaction • Isolating problems • Reactive: Issues
effectiveness – lost to applications and identified when user calls
revenue, high modules • No visibility into
support costs • No insight into each application & operational
• Poor customer user’s experience errors
satisfaction • Providing reports • Time consuming diag.
• No visibility into and visibility to • Inability to anticipate
customer trends stakeholders impact of change

22. Analysts Agree: User Experience is Critical
Consistent end-to-end application and service 87%
performance guarantees
Unplanned infrastructure changes resulting in 73%
incidents and downtime
Unanticipated infrastructure effects from 49%
consolidation & new application projects
Mis-configuration of network objects 28%
Source: Forrester Research, Top Five Challenges For Enterprise IT Infrastructure Managers — And How To Resolve Them, Thomas Mendel, March 2005
“Real life experience shows: 74% of problem alerts come from end users
complaining to the service desk about performance problems, not from
monitoring infrastructure components!”
Jean-Pierre Garbani
Forrester Research 2004

23. TomTom
Delights customers and recovers revenue lost because of IT issues
“There was a point in time when we were more or less
driving in the dark…Now I’m convinced that for us, this
product will pay back within the first six months, easily.”
-Oscar Diele,
Global vice president of e-commerce
• Marketing removed user manual from the box, not telling IT!
Challenge • Product was a big hit last Christmas, many people went on the website to
download user manuals & performance went down, including the online shop
• 27% of customer about to check out canceled their transaction
• Oracle Enterprise Manager
Solution – Real User Experience Insight
• IT gets alerted, understands and reacts very quickly, cutting down the 27% loss.
• IT captures all the 27% customers emails , Marketing immediately sends an
Benefits email with an apology and a coupon to attract those customers to shop again.
• TomTom recovered most of the 27% loss
http://www.oracle.com/profit/features/091908_tomtom_.html

24. Solution Overview
Optimizing end-user experience through real end-user monitoring
Business Reporting
KPI Monitoring Data Export
Real User Experience Insight
key highlights:
- Replay of poor performance
Transaction Performance - Executive dashboards Executive dashboard
- Extensive KPI and SLA
monitoring
- Full alerting capabilities
- Quick bottle-neck analyses
- Transaction performance
analyses
- Customized reporting
- Trend analyses
- Full data integration via XML

25. Real User Experience Insight For:
Line of Business executives
Satisfaction Reports Business Transaction Funnel
Usage Reports:
• Data Browser
• Sales by
Country/State

26. Real User Experience Insight For:
Application Owners and Application Support
Dashboards
& SLM
Business Service Monitoring User Session Tracking,
(e.g. Order Entry, Buy Stock) Page
Replay

27. Real User Experience Insight For:
IT Operations
KPI configuration,
monitoring, &
alerting
Server, network,
& object
performance
diagnostics
Data Export
Error reports
& analysis

28. Application Performance Testing & Management
Real User Application
Experience Insight Testing Suite
Enterprise
Manager
Application Lifecycle

29. Application Operations Management
Top Challenges & Risks
• How to identify and resolve
business issues?
• How to manage entire
application lifecycle and
reduce compliance risks?
• How to reduce operational
costs?
• How to prioritize issues that
impact business user
experience?

30. Solution:
Integrated, Top-Down Application Management
Application
Users
Application Siebel, PeopleSoft, EBS… Java, .NET, Composite Exchange, Legacy
Software
Oracle Applications Custom Applications Non-Oracle Applications
Application Web Server, Portal Database AS, ESB, SOA
Components Presentation Layer Management Systems Middleware
Infrastructure
Components
Virtual Environments Servers Databases Storage
Oracle Enterprise Manager

31. Comprehensive Applications Management
Oracle Enterprise Manager
• Manage Oracle applications
and technology from a single
console
• Application Management Packs for
E-Business Suite, Siebel, PeopleSoft,
JDE
• Optimize User Experience for
Oracle Applications
• E-Business Suite Accelerator
• PeopleSoft Accelerator
• Siebel Accelerator

32. Oracle Application Management Packs
Complement Application Tools
Application Management Pack Application Management Pack Application Management Pack
for PeopleSoft Enterprise for Siebel for Oracle E-Business Suite
Server Manager
PS/Admin Oracle Applications Manager
SARManalyzer, SARMquery
Performance Monitor iSetup
Diagnostic Data Collector
PeopleTools Rapid Install
Environment Verification Tool
Change Impact Manager Rapid Clone
Application Deployment Mgr

33. Application Operations Management
Managing Packaged Applications
Oracle E-Business Suite – PeopleSoft – Siebel
E-
Common Capabilities
• 1-step Discovery • Out-of-box transactions • Reports
• Topologies • System monitoring • Configuration Management
• Service Level Management • Application monitoring • Transaction Tracing
• Root-cause/Impact Analysis • Dashboards • Out-of-box service models
Embedded Application-Specific Capabilities
E-Business Suite PeopleSoft Siebel
• Topological root-cause views • 500+ config metrics & • 50+ business metrics
• 100+ configuration metrics PSAdmin admin tasks • Out-of-box processes
• Clone automation for test • 25+ “tribal” knowledge- • HI Component support
production environments based policies
• SARM integration – real
• Cross-instance comparisons • Centralized Log Mgt end user metrics,
• Key bus flow monitoring transaction diagnostics

34. <Insert Picture Here>
Application Management Pack
for Oracle E-Business Suite

35. Application Management Pack for EBS
E-Business Suite-Specific Management
• Full Visibility into E-Business Suite
Components
• E-Business Suite System Parameter
Change Tracking
• Cloning Automation
• Seamless Integration with vs
Oracle Applications Manager
• Support R12 and R11i ATG RUP4 or
above

36. <Insert Picture Here>
Benjamin Cabanas
Program Manager
GE Infrastructure
“Implementing Application Management Pack
for Oracle E-Business Suite has allowed
GE Infrastructure to realize $200K annual
savings, 84% reduction in clone cycle time, and
75% reduction in resources”

37. <Insert Picture Here>
Application Change Management
Pack for Oracle E-Business Suite
(ACP)

38. Application Change Management Pack for
Oracle E-Business Suite: Version 3.0
Simplify Change Improve User Reduce Human
Orchestration productivity Errors

39. Product Overview
Summary of Key Features
Customization Manager Patch Manager Setup Manager
• Create AD compliant • Configurable Patch Process: • Central console for all activity
customization packages Step-by-step interview process
• Migrate functional
• Central console for all custom • Deploys custom and standard configuration/setup data across
patches Oracle patches from a central different instances of EBS
• Custom patches can be applied console
using Patch Manager • Group extracts, loads, and
• Simultaneous, multi-patch, reports into projects for easier
• Over 200 file types supported multi-target deployment management and reuse.
• Creating build lists with • Integration with Customization • Recognize and enforce the
manifests Manager dependencies between different
• Build or compile Java or Form • Multiple Patch Options: Support selection sets and provide
libraries for wide variety of patching sequencing recommendations.
• Validate the custom files based options such as Hot Patching, • Automated, simultaneous loads
on a set of standards disable archive mode….etc across multiple instances
• Interface with Version Control • Centralized Patch Deployment • Perform loads across different
systems (CVS, Subversion) Tracking: In Progress, RUP levels
• Capture additional Metadata Completed, and Scheduled
• Share the extracts, loads, and
around customization • Integrates with MetaLink to reports content with other
• Standard and Comparison download patches users.
reports

40. <Insert Picture Here>
Application Management Pack
for Siebel

41. Application Management Pack for Siebel
Siebel-Specific Management
• Full Visibility into Siebel Server
Components
• SARM Integration
• Enterprise / Server / Server
Component Parameters
• Siebel Business Metrics

42. <Insert Picture Here>
Application Management Pack
for PeopleSoft

43. Application Management Pack for Peoplesoft
Peoplesoft-Specific Management
• Full Visibility into PeopleSoft
Components
• PeopleSoft System Parameter Change
Tracking
• Out-of-box Configuration Policies
• Centralized Domain Administration vs
• Support PeopleTools 8.48 & above

44. Projected Cost Savings
• Reduce User Reported
Incidents to Application
Administrators by Over 50%*
Costs • Reduce Problem Resolution
Times by Over 50%*
• Increase Administrator
Productivity by over 25%*
• 10 – 18 months payback*
* actual savings, productivity improvements, and payback period may vary

45. Summary

46. Application Quality Management
Pre-Deployment Post-Deployment
Performance Testing Performance Monitoring
Real User Monitoring
Real Application Testing • Capture user performance
• Database Testing • Identify key spot in applications
• Database Capture/Replay • Alerting/Reporting
• SQL Queries Capture/Replay
Service Level Management
Load & Stress Testing • Synthetic transactions
• Any web application • Active Monitoring
• Server Performance metrics
• Collaborative testing Application Management Packs
Functional/Regression Testing SOA Application Management
• Onescript creation
• Easy record/playback Diagnostic and Tuning
Test Management Provisioning, Configuration
Management
Data Masking
• Testing prod data securely Change Tracking
and Synchronization
END-TO-END PERFORMANCE MANAGEMENT

47. Application Performance Testing & Management
Real User
Experience Insight
Application
Testing Suite
Enterprise
Manager
Application Lifecycle

48. Oracle Helps You Maximize Customer Value
Deploys SOA Saves 80% time and
Avoids online revenue
infrastructure 92% effort for managing
losses up to 25%
faster Databases
Cuts configuration
Improves IT Drives asset utilization
management effort by
productivity by 25% up by 70%
90%
Saves $1.9 million Saves $170,000 per Replaces manual
with Oracle Enterprise year with Oracle tools with automation;
Manager Enterprise Manager saves time by 50%
Saves weeks on
Reduces Database Reduces provisioning
application testing
testing time by 90% effort by 75%
time
Cuts application Delivers 24/7 uptime
Reduces critical
testing from weeks to with Oracle Enterprise
patching time by 80%
hours Manager

49. Next Steps

51. Architecture: Network Protocol Analysis
Collected data: Page
request
•Client-ip, Server-ip, Gateway-ip
Network Response Time
•Requested URL (Page,Object)
•Cookie/Session-id Internet
•Referrer
•GET & POST requests
• User agents (phone type)
Firewall
Sniffing Switch
Server Response Time
•Response?
•Server error, web site
Web Web Web
error, Hit OK
server server server
•Content of page (i.e order
details, functional errors)
•Delivered?
•Aborted? DB DB
•Network timeout?

52. Key Customers
250+ … and growing