At Atlassian, support is essential to a successful customer experience. And, because of it's customer-facing position, support is also a great resource for product teams to continually use customer feedback to improve their product or service. Creating strong customer feedback loops means you are building a better experience for your customers day by day.
In this talk, you'll learn 5 ways Atlassian tools facilitate the relationship between support and development and how you can establish these processes and metrics in your own organization.
This talk is a follow on from "How Atlassian Support & Development Team Up to Release Software" from Summit 2016.
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
How Atlassian Broke Down the Walls Between Support & Development
1. How Atlassian Broke Down the Walls
Between Support & Development
MATT SAXBY | SERVICE ENABLEMENT TEAM LEAD | ATLASSIAN
MATT HUNTER | SERVICE ENABLEMENT TEAM LEAD | ATLASSIAN
18. SHIRT VOTE
Add a new shirt
design
View existing shirt
designs ordered by
most votes
JIRA Service Desk
2015
ShipIt 30
2015
40,000 Customers
2014
Game of Codes
2014
200 votes
180 votes
120 votes
80 votes
Need Help?
19. SHIRT VOTE
Full screen
images giving
you an up close
and personal
look at the shirt
Description with all
the detail you would
need
JIRA Service Desk
2015
Vote
Designed to promote JIRA
Service Desk and give a great
way for users (both inside and
outside Atlassian) able to pro…
Ability to vote
for the shirts
you love
200 votes
21. How do you tell if a product
is causing customer pain?
22. Things to
Consider:
Actionable
You can tell what has caused
customer pain to go up.
Representative
When the number goes up customer
pain goes up.
Trackable
Easy to see current status when
needed
23. In App
How to create tickets?
1 2
support.shirtvote.com
27. Things to
Consider:
Actionable
You can tell what has caused
customer pain to go up
Representative
When the number goes up customer
pain goes up
Trackable
Easy to see current status when
needed
32. Contact Index Over Time
5
6.4
7.8
9.2
10.6
12
January February March April May June July August
33. Things to
Consider:
Actionable
You can tell what has caused
customer pain to go up
Representative
When the number goes up customer
pain goes up
Trackable
Easy to see current status when
needed
35. Android App
I can't view shirts
When I open the app to view the list of shirts, it comes up with an error
36.
37. = 15 Voting on Shirts
Data Conclusion
= 27 Viewing Shirts
= 12 Adding Shirts
= 6 Searching for shirts
38. Things to
Consider:
Actionable
You can tell what has caused
customer pain to go up
Representative
When the number goes up customer
pain goes up
Trackable
Easy to see current status when
needed
42. Bug Ranking
Too much time was being wasted
deciding which bugs we should solve
first.
We needed a system to consistently
and objectively rank bugs, agreed on
across the business.
48. 10 customers x
3 active users avg
20 customers x
1 active users avg
VIEW SHIRTS BUG VOTING ON SHIRTS BUG
20 active
users total
30 active
users total
50. Major
Performance problems or
breaks minor functionality
Severity
Minor
Inconvenience such as a
display problem or an
inconsistency
Critical
Causes application outage,
or breaks core functionality
8 4 2
51. 30 active users impacted20 active users impacted
VIEW SHIRTS BUG VOTING ON SHIRTS BUG
Causes
outage
Minor
feature not
working
52. 30 active users impacted20 active users impacted
VIEW SHIRTS BUG VOTING ON SHIRTS BUG
Critical
Severity
Major
Severity
54. Customers Impacted Per MonthCustomers
0
1
2
3
4
Jan Feb Mar Apr May June July
View Bug Voting Bug
55. 30 active users impacted
Major Severity
20 active users impacted
Critical Severity
VIEW SHIRTS BUG VOTING ON SHIRTS BUG
2 reports in
last 90 days
10 reports in
last 90 days
56. Severity
How bad is the impact of the
bug?
User Impact Score
Recency
Is the bug getting better or
worse?
Impacted Users
Number of users impacted
by a bug
58. 30 active users impacted
Major Severity
10 reports in last 90 days
20 active users impacted
Critical Severity
2 reports in last 90 days
VIEW SHIRTS BUG VOTING ON SHIRTS BUG
59. 30 (users)
x
4 (severity)
x
10 (recency)
20 (users)
x
8 (severity)
x
2 (recency)
VIEW SHIRTS BUG VOTING ON SHIRTS BUG
= 320 = 1200
62. KEY SUMMARY UIS
BUG-4 Images not displaying 7
BUG-3
BUG-1
Shirts not showing in correct order 4
Viewing shirts crashes application 3
BUG-2 Can’t upload images 2
63. KEY SUMMARY UIS
BUG-4 Images not displaying 7
BUG-3
BUG-1
Shirts not showing in correct order 4
Viewing shirts crashes application 3
BUG-2 Can’t upload images 2
BUG-5 Voting on shirts reduces vote count 12
64. After
Alignment on which bugs are the most
important, bug master always grabs the
issue with the highest UIS.
Before
Frequent disagreement on the highest
bug priorities and what to work on next.
66. SHIRT VOTE
JIRA Service Desk
2015
ShipIt 30
2015
40,000 Customers
2014
Game of Codes
2014
200 votes
180 votes
120 votes
80 votes
67. SHIRT VOTE
JIRA Service Desk
2015
ShipIt 30
2015
40,000 Customers
2014
Game of Codes
2014
200 votes
180 votes
120 votes
80 votes
Votes
Title
Date
Sort shirts in the
order that suits you
best
68. FEATURE LIFECYCLE - THE OLD
Feature is
developed
Code is reviewed
and tests are run.
Change is rolled
out to entire
customer base
Bugs related to the
feature that weren’t
caught during
review are fixed.
Dev Review Rollout Fix
69. Gradual Rollouts - The Old
0
13
25
38
50
Rollout%
0%
25%
50%
75%
100%
Week 1 Week 2 Week 3 Week 4 Week 5
% of Customers With Feature Support Tickets
70. SHIRT VOTE
JIRA Service Desk
2015
ShipIt 30
2015
40,000 Customers
2014
Game of Codes
2014
200 votes
180 votes
120 votes
80 votes
Votes
Date
Title
72. Gradual Rollouts - The Old
0
13
25
38
50
Rollout%
0%
25%
50%
75%
100%
Week 1 Week 2 Week 3 Week 4 Week 5
% of Customers With Feature Support Tickets
73. Gradual Rollouts - The Old
0
13
25
38
50
Rollout%
0%
25%
50%
75%
100%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
% of Customers With Feature Support Tickets
74. Gradual Rollouts - The New
0
12.5
25
37.5
50
Rollout%
0%
25%
50%
75%
100%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7
% of Customers With Feature Support Tickets
75. Gradual Rollouts - The New
0
12.5
25
37.5
50
Rollout%
0%
25%
50%
75%
100%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7
% of Customers With Feature Support Tickets
76. Gradual Rollouts - The New
0
12.5
25
37.5
50
Rollout%
0%
25%
50%
75%
100%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7
% of Customers With Feature Support Tickets
77. BIG BANG VS GRADUAL ROLLOUT
0
2.5
5
7.5
10
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7
Contact Rate - Gradual Contact Rate - Big Bang
Tickets Created:
Big Bang - 57
Gradual - 29
78. FEATURE LIFECYCLE
Feature is
developed
Code is reviewed
and tests are run.
Change is
gradually rolled
out.
Most bugs were
caught during
rollout, so this
feature is ready to
close off.
Dev Review Rollout Close
Tight feedback
loop to ensure
problems are
caught early
Feedback
79. Contact Rate
Create a contact rate
specifically monitoring
tickets received from
customers with the feature.
Gradual Rollout Pro Tips
Feedback Loop
Work closely with the
Development team to share
feedback asap.
Readiness
Involvement with
Development early in the
cycle ensures the support
team is ready for the change.
81. Hiring
Allows you to
forecast the number
of Supporters you
need to hire
Customer
Experience
Helps to tell the
customer experience
story
Development
Negotiations
Assists with
negotiating
Development
priorities
Goal Setting
Sets goals to track
the success of the
team.
Why is prediction Important?
82. Baseline
Forecast of support contact
assuming current trends
continue
Initiatives
Changes impacting product
that are expected to impact
support contact
How do you predict?
83. Baseline
Forecast of support contact
assuming current trends
continue
Initiatives
Changes impacting product
that are expected to impact
support contact
How do you predict?
84. Complex Modelling
• Prophet
• ARIMA
BASE FORECASTING
Basic Modelling
Average Yearly Contact Index
0
3
6
9
12
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
FY16 FY17
85. BASE FORECASTING
0
3
6
9
12
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
FY16 FY17 Average
Complex Modelling
• Prophet
• ARIMA
Basic Modelling
Average Yearly Contact Index
86. BASE FORECASTING
0
3
6
9
12
Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Average
Complex Modelling
• Prophet
• ARIMA
Basic Modelling
Average Yearly Contact Index
90. Baseline
Forecast of support contact
assuming current trends
continue
Initiatives
Changes impacting product
that are expected to impact
support contact
How do you predict?
93. CONTACT DECREASE
0
20
40
60
80
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Ticket Forecast Post Bug Fix
Shirt voting bug due to be
fixed in February.
95. SHIRT VOTE
JIRA Service Desk
2015
ShipIt 30
2015
40,000 Customers
2014
Game of Codes
2014
200 votes
180 votes
120 votes
80 votes
Search for the shirts
that interest you
Search
101. Hiring
Allows you to
forecast the number
of Supporters you
need to hire
Customer
Experience
Helps to tell the
customer experience
story
Development
Negotiations
Assists with
negotiating
Development
priorities
Goal Setting
Sets goals to track
the success of the
team.
Why is prediction important?
102. Insights
Making things actionable is
fundamental to creating
change
In Summary
Prediction
You need to look to the
future to prepare your
support team and ensure
they can provide legendary
support
Reporting
Have a Trackable,
Representative and
Actionable way to gauge
customer impact.
103. Thank you!
MATT SAXBY | SERVICE ENABLEMENT TEAM LEAD | ATLASSIAN
MATT HUNTER | SERVICE ENABLEMENT TEAM LEAD | ATLASSIAN