Customer experience measurement in the utilities industry – closing the loop

Utility Customer Experience
Measurement – Closing the Loop
April 12, 2016
Shaikat Sen, Senior Vice President, Blueocean
Market Intelligence
Larry Simpson, Research Principal, Pacific Gas
and Electric Company
© 2016 Blueocean Market Intelligence 1

Agenda
From Rating Scales to Rate Cases: The case for CEx Measurement
in the Utility Industry
CEx 360 – A Holistic Approach
Identifying Touchpoints/Pain Points
Monitor/Track Performance
Segmentation/Modeling
Cadence
The Utility Perspective
Final Thoughts

From Rating Scales to Rate Cases: The
Case for CEx Measurement in the
Utility Industry

Higher Levels of Customer Satisfaction Positively
Impact Credit Ratings, Operating Margins, and ROE
Higher Customer
Satisfaction
In 2005, S&P clarified that
their methodology for
determining risk and
credit ratings for utilities
would include the
customer satisfaction
score, along with a host of
other metrics
Credit
Ratings
In 2011, J.D. Power found
“regulated US Utilities
that achieve the highest
customer satisfaction in
J.D. Power studies also
reported the highest rates
of net operating margin to
FERC”
Operating
Margins
J.D. Power and Associates
also found that higher
customer satisfaction
scores were correlated
with more favorable rate
case decisions
Return on
Equity

Expectations are Increasing, but Meeting
Expectations Can Strengthen the Future
• Unprecedented impact of
social media
- Customer satisfaction
has risen to the top of
consumer
consciousness
• Public exposure to
benchmarking studies (J.D.
Power)
- Customers are seeing
how other utilities are
being perceived by their
customers
• Customer satisfaction as a
competitive differentiator
− Critical when seeking
growth through the
development of new
unregulated products
and services, or
protecting the core
business from
disruptive entrants
• Better customer
experiences
− Cornerstone for positive
outcomes around co-
created energy
efficiency solutions

Utilities Ranked 9 Out of Top 14 Industries on Customer
Satisfaction, Trust in Utilities is on the rise.
In a worldwide study conducted in 2014, Accenture found that trust in utilities had increased by 13% to 37% in
2014…up from 24% in 2013.
Expectations from Utility Companies (2014)
8.59
8.52
8.41
8.35
8.28
8.20
7.89
7.88
7.82
7.73
7.71
7.71
7.66
7.55
Customer Engagement | McKinsey on
Marketing & Sales
1 Brokerage
2 Auto Ins
3 Retail
4 Bank
5 Hotel
6 Health Providers
7 Mobile Phone
8 Airlines
9 Utilities
10 Phone
11 Postal
12 Health Inc
13 Internet
14 Pay TV
Rank
2012
Industry
CSAT
average score1
2011-2013
trend
1 Customer satisfaction was measured on a scale of 1-10; includes top 3 companies per industry
per respondent. Consistency of experience operationally defined as standard deviation, that is,
consistency of perception across surveyed demographic
Simple home improvement products and materials to
save energy
66%
Home energy audits/consultations to help save
energy
62%
Home energy generation products 58%
Devices or services to automate home energy
management
56%
Installation and/or maintenance services for home
energy devices
56%
Backup energy storage or power generator in case
energy goes out
53%
Warranty and/or financing plans for home energy
devices
52%
Source: The New Energy Consumer: Architecting for the Future, Accenture, 2014.

CEx 360 –
A Holistic Approach

Components of an Effective 360 CEx Measurement
Program
Insights

Information Sources to Tap Into
• Qualitative
• Explore customer journeys
• Identify touchpoints
• Assess pain points
• Quantitative
• Relationship surveys
• Transactional surveys
• Blogs/forums/discussions
• Complaint/review web sites
• Facebook/Twitter/other sites
• Segmentation data
• J.D. Power results
• Other benchmarking
studies
• Rate plan participation
• Bill payment history
• Energy usage
• Demographics
• Call center transcripts
• Customer emails
• Customer/field service
agent updates

Mapping Information Sources to the 360 CEx Cycle
Implement
Changes
Communicate
with Customers
Segmentation/
Modeling
Transactional Data/
3rd Party Data
Identify Touch/
Pain Points
Qualitative Research/
Unstructured
Feedback/Social Media
Monitor/Track
Performance
Quantitative Surveys/
Benchmarking Surveys

Identifying Touchpoints /
Pain Points

Harvesting Information…
Enumerate, identify and map touchpoints through customer focus groups
and journey mapping sessions with internal stakeholders
Monitor relevant industry news through clipping services, RSS feeds, etc.
Systematically listen to and harvest
information from as many customer
information sources as possible
Recordings Transcripts Emails Field Notes
Adopt one or more social media monitoring tools to
“scrape” data from a multitude of social media websites
Invest in text analytics solutions to analyze
unstructured data streams

…And Harnessing It
Siphon all feedback to a dedicated “Listening
Intelligence Team”
Code and monitor incoming information on an ongoing
basis
Develop a standardized code frame to organize
unstructured information across all data sources
Generate ongoing weekly/monthly reports highlighting
top customer issues

Visualization Examples of Unstructured Data
Theme Comment
Overall Service/
Feedback (+)
• No problems/very few power outages/service is
good/happy with service
• Good company/honest/give back to community
Overall Service/
Feedback (-)
• Frequent outages/Service is not reliable
• Service is average
• Not aware of any other services that are
provided
• There is always room for improvement/no one is
perfect
• Do not like company
• Heard negative things about company
• Not happy or dissatisfied/neutral
Customer Service/
Field Service(+)
• Customer service (friendly, helpful,
knowledgeable, etc.)
• Respond quickly to service issues
Customer Service/
Field Service (-)
• Customer service (hard to reach reps, not
helpful, etc.)
• Do not respond quickly to service issues
Billing/
Website mention (+)
• Accurate billing/timely
• Online billing available
• Easy to pay bills/easy to read bills
• Can see/compare energy usage
• Website mentions/easy to use
Billing/
Website mention (-)
• Billing issues
• Website issues (confusing, no online billing)
Rates/Discounts (+) • Lower rates/discounts

Monitoring/Tracking
Performance

Value of Quantitative Measurement
Quantitative
Assessment is
the Backbone
of a Strong
Measurement
Program
• Results are representative of your overall customer base
• Identifies specific areas where you are doing well and those needing
attention
• Helps to prioritize your organizational focus to get the best ROI
• Perception is reality – you need to keep your finger on the pulse of your
customers’ perceptions
Components
of a 360
Program
• Transactional surveys
• Relationship surveys
• Competitive benchmarking (optional)

Part 2: Develop transactional surveys associated with each customer journey
Part 1: Understand the customer journey associated with each transactional touchpoint
Transactional Surveys – Best Practices
• Conduct customer journey focus groups
− Internal multi-functional stakeholder brainstorming sessions
− Customer focus groups
• Enumerate as many customer journeys that reflect how your customers interact with your
organization
− New customer sign-up
− Tree trimming
− Emergency service response
− Other
• Prioritize customer journeys and develop a strategy and timeline to address each journey
• Keep the survey short
• Focused on the specific customer journey
• Include questions that tie back to high level customer satisfaction metrics
• Deploy after each transaction is complete

Relationship Survey Design – Best Practices
Characteristics of
High Performing Utilities
Positive customer experience
High levels of customer engagement
in products and services offered
Strong communication with
customers
High levels of customer trust in the
brand
Overall Experience
Overall Trust
Overall Communication
Engagement Levels
Ease of Doing Business
Overall Satisfaction
Likelihood to Recommend
Likelihood to Switch
Relevant Key Metrics to Consider

Experience, Trust, and Communications – Example
Metrics
Billing
Payment options
Billing statement
clarity
Billing statement
accuracy
Meter accuracy
Pricing
Fair price
Education - rate
plans
Timely change
notifications
Custom rate plans
E-Efficiency
Education - saving
energy
Energy efficiency
incentives
Competence
Process excellence
Well-managed
Leader
Corporate Values
Keeps promises
Take responsibility
Open
communications
Cares about
employees
Customer First
Values customers
Extra mile
Ease of doing
business
Understands needs
Customer ExperienceCustomer Trust
Communications
Other
Communications
Web site
assessment
Specific marketing
collaterals
Advertising recall
External News
Exposure to news
Favorability/un-
favorability of news
Word of Mouth
Discussions with
others
Positive/Negative
opinions shared

Prioritizing Actions – Best Practices
• Carefully plan the structure and sequence of the questionnaire
• Determine where you want the key metrics to be:
• Keep attributes and related dimensions together, but randomize and rotate dimensions and attributes to
prevent order bias
• Decide on an appropriate rating scale (5-pt., 7-pt., 10-pt.) and stay consistent
• Establish a theoretical model of how attributes might relate to each dimension, and how each dimension
relates to the key outcome measure
• Responses sensitized by preceding assessments
• Statistical models tend to be stronger
• But, results may not reflect reality
…or at the end
At the beginning…
• Responses tend to reflect top-of-mind perceptions
• Results may be more closely aligned with real life
• However, statistical models tend to be weaker

Values customers
Hypothesizing Expected Relationships
Overall
Satisfaction
Trust
Experience
Customer First
Environment
Competence
Billing
E-Efficiency
Reliability
Management
Leadership
Payment options
Statement clarity
Statement accuracy
Extra mile
Ease of doing business
Education on
saving energy
Energy efficiency
initiatives
Process

Driver Modeling – Best Practices
Standard regression doesn’t work well
with attitude and perception data
New techniques are available today to
get around these challenges
• Shapley Value Analysis or
Dominance Analysis or Relative
Weight Analysis
• Impractical to use before because
of the enormous amount of
computing power required
• Today’s computers are able to run
the required hundreds of
thousands of regressions in a
matter of hours
• Traditional techniques can fail
when predictor variables are
correlated with one another or
have skewed distributions
• Attitude and perception data are
almost always correlated and
skewed

Relative Importance of Drivers
Standard Regression Model Shapley/Dominance Analysis
R2= 60%
Var. Important %
Predictor 1 = 15%
Predictor 2 = 20%
Outcome - dependent var.
Predictor 1 , Predictor 2 - Independent var.
Predictor 1
Predictor 2
Outcome
40%
15%
25%
20%
Multicollinearity
Outcome - dependent var.
Predictor 1 , Predictor 2 - Independent var.
Predictor 1
Predictor 2
Outcome
40%
15%
20%
Multicollinearity
25%
21% 4%
R2 = 60%
Var. Important %
Predictor 1 = 36%
Predictor 2 = 24%

2%
7%
11%
21%
25%
34%
Field ServiceCustomer
Service
BillingE-EfficiencyReliabilityPricing
Relative Importance
10%
16%16%
18%
23%
26%
Corporate
Values
EnvironmentCompetenceCommunityCustomer
First
Safety
Relative Importance
Overall Satisfaction
Customer Trust
40%
Customer Experience
60%
R2=80%R2=70%
© 2016 Blueocean Market Intelligence
Relative Importance of Dimensions
24

Customer Experience
60%
Pricing E-Efficiency Reliability Billing
Customer Service
7%
Field Service
11%
14%
16%
17%
19%
25%
Customer Service Representative
Problem resolution
Overall time
CSR Competence
CSR Communication
CSR Courteousness
Phone system navigation
Phone system navigation
Overall time
Problem resolution
R2 = 78%
R2 = 80%
Live (70%) Automated (30%)
31%
30%
39%
Automated Customer
Service
© 2016 Blueocean Market Intelligence
Relative Importance of Attributes
25

Sampling – Best Practices
Target Audience
• Residential surveys – head of household/spouse/partner, etc.
• SMB surveys -
 One account can have multiple sites; same company can have multiple accounts
 Ideally, your population should be defined as an account and site combination that has a unique account
contact
• Minimum recommended sample size for a
segment is about 400
• However, you have to consider all relevant
segments
• Recommended minimum sample size for
each sub-segment is 100 – or more – if
budget allows
Sample Size Sample Sizes and Error Margins
Margin of error
(Precision)
Confidence Level
90% 95% 99%
3.0% 747 1067 1849
5.0% 269 384 666
10.0% 67 96 166

Data Collection Options
Advantages Disadvantages
Channels
Outbound Phone
Inbound Phone/
SMS Surveys
Email
Regular Mail
Web-based
App-based
• Expensive
• Cellphone prevalence making it more and
more difficult and expensive
• Refusal rates are increasing
• Tends to skew towards older, women
• IVR programming required (inbound phone)
• Need to automate survey offering to
prevent bias (inbound phone)
• Surveys need to be very short
• Response rates are very low
• Need to have email addresses
• More expensive than email
• Very low response rates
• Usually takes longer in field
• Requires internet awareness and may skew
sample to a younger demographic
• Customer has to come to the web site to get
an opportunity to participate
• Still in infancy, but gaining popularity
• Low uptake for older generation
• Easy to complete
• Easy to control
• Relatively high response rate
• Immediate response – feedback is more
accurate
• Generally good response rates
• Inexpensive after initial set-up
• Relatively cheap and easy to set up
• Cost effective to run continuously
• 80-90% of responses are in within 72 hours
• Generally cheaper than telephone
• Can capture good, detailed feedback
• Often used to ‘rate’ content
• Useful for web site assessments
• Typically used in conjunction with email
surveys
• Emerging approach
• Cost effective to run continuously
• “In”

Data Collection – Best Practices
Mixed/hybrid surveys (landline, cellphone, and email) may be optimal
– Acceptable for ongoing tracking programs
– Allows you to capture a more representative sample of your customers
– Happy medium in cost and ability to leverage existing customer
information
Customer Experience
60
65
70
75
80
Q2 '14 Q3 '14 Q4 '14 Q1 '15 Q2 '15 Q3 '15 Q4 '15
Phone CONTROL Phone TEST Online TEST Combined TEST
• Ensure sample
composition
matches
distribution of
landline, cell phone,
and email addresses
available in the
customer database.

Segmentation and Modeling

Leveraging Transactional/Syndicated Data to Profile,
Understand and Predict Customer Behaviors
Utilities have an enormous amount of data on their
customers
Allows you to associate satisfaction with factors like energy
use, monthly bills, household composition, and so on
Matching back quantitative survey data to customer base
can yield deeper insights about attitudes and behaviors
Attitude and perception data allows modeling of behaviors
and segmentation to maximize produce development and
communication opportunities

Developing a Customer Engagement Metric
(selects EE programs &
renewable programs)
Management of
energy usage
Adoption of
new technology
Takes advantage
of offers
Interactions
with Utility
(signs up for MyAcct,
eNotifications, surveys)
(selects TOD programs,
specific eNotifications)
(campaign respondents,
enrolls in BPCC programs)
“Dimension-
level”
engagement
score
Overall
“Engagement
Score” -
weighted average
of dimensional
engagement
scores
50.30%
12.90%
17.60% 19.20%
Adopt New
Technologies
Interaction with
Utility
Manages energy
usage
Takes Advantages
of Offer
Composite dimension weights

The Customer Engagement Continuum
Focus Areas
.00 13.90 41.70 69.51 97.31 125.11 152.91 180.70 208.52 236.32 264.12 291.93 319.73 347.53 375.33 403.14
27.80 55.60 83.41 111.21 139.01 166.81 194.62 222.42 250.22 278.02 305.83 333.63 361.43 389.23 417
Poorly engaged Moderately engaged Highly engaged Very Highly engaged
Poorly
engaged
Moderately
engaged
Highly
engaged
Very highly
engaged
Interaction with Utility
Managing Energy
Takes Advantages of offers
Adopt new technology
Low Medium High
2% 13%
26%
54%
5%
Very highly
engaged
customers
Highly Engaged
customers
Moderately
engaged
customers
Poorly
engaged
customers
Non-engaged
customers
Segment Size

Type of Dwelling (Single Family)
Definite Home Owner
Household Age (>45 yrs.)
Months at Residence (>82 mths.)
House Square Footage (>1920 sqft)
Pool at home
Pool Area (>100 sqft)
Home Construction quality (good)
Customer tenure (10+ yrs.)
HH Income Range ($75K+)
Married (Husband & wife)
with/without child
Number of People in HH (4+)
Demographics, Attitudes, Energy Use
Home
92%
84%
71%
63%
49%
32%
32%
31%
69%
49%
48%
24%
Total
77%
64%
61%
41%
36%
18%
18%
18%
52%
32%
33%
17%
Household
1%
1%
2%
3%
4%
4%
6%
7%
9%
10%
12%
19%
22%Green Investors
Payback Investors
Green Activists
Show-me Participants
First Costers
Hands-On Believers
Pragmatists
Unplugged
Creatures of Comfort
Budget Watchers
Living in the Now
Tech Frontiersmen
Tech to Live
22%
26%
34%
40%
62%
65%Affordability level
Green affinity
Info action orientation
Investment capacity
Technology propensity
Comfort consumption
13%
12%
9%
11%
17%
5%
4%
10%
4%
5%
6%
2%
5%
40%
52%
26%
23%
26%
15%
EnergyConsumerDynamicsAxciomVariables
Total
1440.3
1159.5 274.1 $161.1
Total (avg.) OffPeak (avg.) OnPeak (avg.) Enrgy Bill (avg.)
kWh
kWh
1231 kWh 902 kWh 170 kWh $142Total
Last 12 Month Energy Use and Bill
Total (avg) Off Peak (avg) On Peak (avg) Energy Bill (avg)

Engagement Segment Profiling
Very Highly Engaged Average Engagement Score: 318/1,000
Higher income, likely to spend more and invest more, stay in larger houses with pools.
Innovators by nature and pro-active. Display high affinity for comfort consumption and
environmental friendliness.
Size: 1.8%
Highly Engaged Average Engagement Score: 191/1,000
Likely to be home-owners who are financially stable and have high disposable incomes.
Tend to be older customers. In general they also have an affinity for comfort
consumption and environmental friendliness.
Size: 13%
Moderately Engaged Average Engagement Score: 110/1,000
Average income, middle aged families with children staying with more no. of people in
HH. Have a pragmatic approach and high info/action orientation. Tend to be innovative,
green friendly, and prefer comfort consumption.
Size: 26%
Poorly Engaged Average Engagement Score: 43/1,000
Tend to be newer customers with less energy consumption having poor credit histories,
staying in smaller apartments. Mainly young single individuals, tech savvy and mobile,
yet to be anchored in long term housing . Generally have lower household incomes.
Size: 54%
Not Engaged Average Engagement Score: 0/1,000
Single wanderers – prefer living in apartments/town houses and usually not for more
than 2 years. Financially cautious with a lower propensity for technology and green
friendly initiatives. Very low on comfort consumption.
Size: 5%

Rules to Predict High Engagement Levels

Cadence

Customer Experience Measurement - Cadence
Month
1 2 3 4 5 6 7 8 9 10 11 12
Ongoing
 Unstructured (text) analysis
 Transactional Surveys
 Relationship Surveys
Ad Hoc Deep Dives/Drill Downs
 Behavioral Connections/Modeling
 Dip-stick Quantitative Surveys
 Qualitative Deep Dives/Ethnographics
 Competitive Benchmarking Survey

The Utility Perspective

Utility Perspective
Evolve research to changing environment
while maintaining essential trending
Adapt to measure emerging customer
needs and obtain feedback about major events
Complement syndicated research with
proprietary surveys
Own survey content, manage changes to
methodology, identify respondents for data
appends and segmentation, and holistically
measure gas and electric satisfaction
Partner with data science to set key
goals and forecast performance
Process has grown from analyzing trend data
and using management judgement to
employing advanced statistical modeling and
Monte Carlo simulation
Supplement perception research to
add clarity
Deep dive into customer touch points using
transactional surveys, qualitative research, and
operational data
Utilize key research to inform the
Customer Experience (CX) team
Collaborate to measure and analyze data, identify
and prioritize friction points, work cross-
functionally, and inform strategic decision making

Final Thoughts
Leverage existing assets (harness all available unstructured/internal data)
Identify a relevant and comprehensive set of attributes to measure
Ensure that the instrument is designed to allow for statistical analysis of key
drivers of overall satisfaction
Create enthusiasm and accountability within your organization
Deploy transactional surveys to monitor customer journeys
Develop a tracking survey instrument that can remain stable over a period of one
to three years

United States | United Kingdom | India | United Arab Emirates
www.blueoceanmi.com
Thank You
Shaikat Sen
Senior Vice President
ssen@blueoceanmi.com
Larry Simpson
Research Principal
Pacific Gas and Electric Company

Customer experience measurement in the utilities industry – closing the loop

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Customer experience measurement in the utilities industry – closing the loop

Ähnlich wie Customer experience measurement in the utilities industry – closing the loop (20)

Mehr von Course5i

Mehr von Course5i (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Customer experience measurement in the utilities industry – closing the loop