Since its beginning, the Performance Advisory Council aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing. During this event, 12 participants convened in Chamonix (France) exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.
3. What I will cover today
• The challenge of getting visibility
• What a good performance management platform looks like
• Real-world challenges and constraints
• Altersis Performance approach to building the platform
• Our methodology, the UMA
• Some examples of solutions we have built
4. Today’s challenge
• Many large companies are finding themselves at a disadvantage
• Performance is part of this story, as it strongly affects agility
• Companies are still struggling to get visibility at the level that matters
• Difficult to move fast in industries like banking and insurance
5. Example: Insurance Company
• Large multinational company: thousands of employees, millions of customers
• Fragmented, silo solutions for performance and capacity management
• Reactive approach to manage the performance of business-critical applications
• Inefficient root-cause analysis for performance issues
6. Our approach
• Each organisation has a different situation and needs
• Evaluate what you have now and where you want to get to
• We use our «Performance Maturity Assessment» for this
• Ensure a stable foundation – good performance testing and APM tools
• Build a service-based view
7. Startingpoint – PerformanceMaturityAssessment
Reactive/Initial
Partially proactive
Proactive
Governance
Mastering
√ No or basic
system/infrastructure
monitoring
√ No or sporadic
performance test
Performance & Stability
Incidents
High costs to fix late
detected performance
and stability defects
Loss of trust, frustration
of users and staff
√ APM for few critical
applications
√ L&P Test for few
critical applications
High costs to manage
many standards and
heterogeneous
monitoring landscapes
High risk for
undetected
performance defects
during development
√ Performance QA, L&P
Test and APM
standardized and
institutionalized
No or suboptimal
usage of the standards
Usage and execution
of the APM standards
not monitored
√ Performance QA, L&P
Test and APM processes
KPI’s are measured
√ Performance QA, L&P
Test and APM standards
are maintained
Execution feedbacks
and experiences not
considered
Performance
management standards
outdated in comparison
with IT industry
evolution and best
practices
√ Continuous Improvement
of Performance QA, L&P
Test and APM processes
and their impact
Sponsor and execute
improvement plans0
1
2
3
4
8. Essentialelements for successful adoption
APM & CQA GOVERNANCE:
Leadership, functions, well-defined service catalogues and processes
ADDRESS THE FIVE CHALLENGES:
Speed of change
Release management
New tools resistance
Skills requirement
Organisation structure
APM & CQA Governance: (Altersis Performance definition)
The leadership, functions, well-defined Service Catalogues and related processes required for a sustainable and efficient coordination between the IT Stakeholders, organizational units, processes and standards in order to continuously assure application quality and performance objectives in production.
9b - Overcome the 5 challenges for Dev and Ops
(i) Rate of change
- Development Teams are still struggling with Continuous Integration, static code analysis and testing
- Operations Teams are still struggling with Event Management and Service Modeling
(ii) Release management
- Continuous Performance roll-out in large organization requires mature Configuration and Deployment Management
- APM roll-out in large organization requires mature Configuration and Deployment Management
(iii) New tools
- Adoption of continuous APM means introducing new technologies & efforts to the already “expensive” and “fragmented” testing stack
- Adoption APM means introducing new technologies and administration costs to the already “expensive” and “fragmented” Monitoring stack
(iv) Skills
- Continuous Performance requires another skills set for Dev QA
- APM requires another skills set for Ops
(v) Org structures
- Continuous Performance roll-out requires a close collaboration between Dev QA and Ops Monitoring
- APM roll-out requires a close collaboration between apps/Dev and Ops
The layer consists of 3 building blocks:
Service modeling and business rules: containing service model definitions and SLAs. Typically data is stored in a Configuration Management Database (CMDB)
A central performance database (PMDB)
A central event console
Typically composed of the following parts:
Analytics, Advanced Diagnostics and Intelligent Alarms
Capacity Planning and Workload Placement
Provisioning and Deployment Automation
This layer is used to give different roles/organization units different views of the underlying monitoring data.
The visualizations are adapted to the respective use-case and tailored to the users.
This also results in very different forms of presentation (e.g., table, graphs).
The Unified Monitoring Architecture (UMA) - a high-level blueprint for implementing a unified and comprehensive architecture for monitoring infrastructure, applications, and services.