January, 2019 North Coast HDI Chapter Meeting. Learn more on our website: www.ncchdi.org

1. xMatters
Major Incident Management
Best practices
Real world case studies
Confidential and Proprietary

2. Hi…my name is Anne Deming
ademing@xmatters.com
Confidential and Proprietary 21/18/2019

3. How many MIM’s a month?
Confidential and Proprietary 31/18/2019

4. A recent survey on Major Incidents
•32% - once a year or less
•42% - at least several times a year
•13% - at least once a month
•13% - at least once a week
Confidential and Proprietary 41/18/2019

5. Must haves – we won’t bury the lead
•Stakeholder Alignment
•Rapid Engagement
• Determine who and when to engage Identifying technical vs. management
contacts
•Intelligent Responses
•Multi-modal Support
•Integrations with Operations
Confidential and Proprietary 51/18/2019

6. 6
Thechallengeyou face….(becauseyoursystemsare yourbrand)
Confidential and Proprietary
Poor communicationduringMajor Incidentscan seriouslyimpact IT

7. 7
CommonMajorIncidentCommunicationApproaches
Confidential and Proprietary 7
Manual
notifications
Email and
distribution lists
mass notification
No Precision No Reliability No
Accountability“Kitchen Sink”
approach results
in alert fatigue
Critical insights
slip through the
cracks
Limited follow-
through to
ensure action

8. 8
Poor communicationduringa Major Incidenthas a bigimpact
Extended downtimes
& missed SLA
Loss of
revenue
Employee
burnout
Tarnished
image of IT
Frustrated
customers
Resolvers not held
accountable
Confidential and Proprietary

9. A Case study for a digital communications company
•Digital communications company
•Over 32 million connections to its network
•120 Million calls everyday in the network
• Connects people, families & businesses (their customers)
• Critical connectivity for tracking / payments on a transit system.
Confidential and Proprietary 91/18/2019

10. HIGHLIGHTS on using xMatters
•We transformed the way they manages incidents, with impressive results:
•Reduced the number of clicks to engage stakeholders
•2 minutes to send communications, an 88% reduction
•3 minutes to engage stakeholders, an 85% reduction
Confidential and Proprietary 101/18/2019

11. HOW?
•A mobile app
•Targeted messages
•Automated escalations
•Communications are easy-to-build, easy-to-send
•Templated communications
Confidential and Proprietary 111/18/2019

12. 1212Confidential and Proprietary
There is a better way

13. Preparation - Creating effective communication plans
•Impact of outsourcing within global teams and 3rd party vendors
•Maintaining up-to-date contact info and schedules
•Developing standard communication
• Defining the communication content
• pre-determined templates
• pre-determined engagement timelines for communications
• pre-determine the blast radius (all or nothing is a bad idea)
Confidential and Proprietary 131/18/2019

14. Action – fire fighting
•Activating the communication plan
•Management of the recovery: Leadership roles, technical guidance
•Inspiring rapid response from tech teams
•Communicating effectively with necessary constituencies
•Tracking communication plan status
Confidential and Proprietary 141/18/2019

15. Reflection – the post mortem
•Root Cause Analysis after a Major Incident
•What types of information gathered during a major incident can be
leveraged to avoid recurrence.
•Tracking SLAs
•Aggregating data for compliance/audit purposes
•Remediating contact info errors
•Driving improved performance – identifying weaknesses in the
communication plan
Confidential and Proprietary 151/18/2019

16. A cheat sheet of questions to start the dialogue
• Who to contact and how to contact them.
• Incident Manager…When you assemble folks how does this happen?
• What about getting folks on the call?
• Consider the Staffing schedule: skeleton crew, 8-5 or are you 24x7?
• What is the timeframe for communication with the business?
• What about escalations? How do you handle them? There is MTTE and time to become affective. Some MIMs take 30 hours.
• A culture consideration: How to identify the right people to contact? Is it changing behavior or is enforcement an issue?
• How do you measure if you are overwhelming one resource vs. another are you tracking?
• How many steps are in your MIM process….Commonly we hear a 5 steps process. (Could yours be adjusted?)
• How & when will you engage 3rd party (hosting providers or technology vendors you need to engage) in during a Major
Incident?
• When outsources escalate back to customer….How will you handle SLAs on the customer side?
• Have you considered communication for upgrades or downgrades P2 to P1 or P1 to P2?
• Is there criteria to go straight to MIM, red alert and bypass triage?
Confidential and Proprietary 161/18/2019

17. Thank you….questions?
Confidential and Proprietary 171/18/2019