2. Overview
Introductions
Leveraging the most out of the data you already have
Setting up baselines and real-time KPI dashboards
Making better decisions from your data
Presenting Benefits Realisation in a way the business will
understand
A LateRooms Case Study
Conclusions
2 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
3. A little bit of LateRooms.com history
LateRooms.com was born in Salford, Greater
Manchester in 1999, starting life as an 'on the
day for the day' booking site for unsold rooms.
By 2007 we had joined the world's leading
leisure travel group, TUI Travel PLC. We're still
loving life here in Manchester today, now
offering over 55,000 properties worldwide
with more UK hotels than anybody else.
Who are LateRooms.com?
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects4
And these days you can still book on the day of
your stay, naturally, but also up to a whole year
in advance - to make things nice and flexible for
you.
4. With a lot of people who like to ‘Standup’…
We work in IT
5 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
5. And focus heavily on collaboration and continuous delivery
We’re structured in product teams
6 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
6. Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
9
Leveraging the
most out of the
data you already
have
7. But it’s siloed within departments or tools
Data, data everywhere…
10 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
8. Who will defend it to the death…
And it has different owners
11 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
9. But nobody thinks their data has quality issues
And nothing lines up
12 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
10. To help alleviate the pain
There are a number of tools and techniques we use as
BA’s to identify data within our organisations, how
close it is to meeting our requirements and how
accessible it is.
Some of the techniques we use include:
• Requirements & value analysis
• Stakeholder identification
• GAP analysis
• Benchmarking analysis
BA Activities that we undertake at this stage
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects13
11. Dust off the business case
At some point in your project someone
committed to some benefits that were
expected.
Where there is a benefit there has to be a
measurement:
• Higher sales
• Lower costs
• Greater customer retention
• Improved efficiencies
Requirements & value analysis
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects14
12. Who’s data is it and how can they help
Data is often collected by organisations to
support a number of requirements. While
this data can be repurposed it may not be
fit for purpose. Consider the following:
• Who in your team, organisation or
external to your organisation may
already have data that you can use?
• What was the data originally captured
for and is there any restriction on its use?
• Are there any data sets that could be
used to support what you’re trying to
measure?
Stakeholder identification
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects15
13. Does the data meet your requirements or does it need enhancing?
Once you have identified your data sources it’s
important to work out if they meet your
requirements or if they need to be enhanced to
support your needs.
If your data source represents a subset of the data
needed you may need to decide between
enhancing your data collection (if that is an
option). Alternatively you may be able to combine
data sets if there is a common unique attribute
across both sets (however, this may be a risky
strategy if the data sets get out of sync).
Consider:
• ROI for enhancing
• How “accurate” you need the data to be
• If a combined data set is reliable enough
GAP Analysis
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects16
Measurementperformance
14. Don’t let the data become stale
Once data is collected you should decide on a
strategy to ensure it remains fresh.
• Set regular reminders to review data sources
• Get added to distribution lists to hear about
changes in data collection projects (Business
Intelligence or data warehouse updates?)
• Periodically validate that the data being collected
is still relevant and that it’s collection mechanism
has not been superseded by an alternative means
Benchmark and improve
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects17
15. Setting up baselines
and real-time KPI
dashboards
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
6
16. And measure it continuously
Once you have identified your data
sources, stakeholders and any gaps
you have in it, the next step is to give
it some meaning.
• Define your targets
• Define your thresholds
• Really good
• Really bad
• Know what happens if you breach
a threshold…
• Decide on measurement intervals
and make sure they happen.
Give your data meaning
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects19
17. ..and make sure they’re really big!
Make your dashboards visible
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects20
18. Make it happen
“If Engineering at Etsy has a
religion, it’s the Church of Graphs.
If it moves, we track it. Sometimes
we’ll draw a graph of something
that isn’t moving yet, just in case it
decides to make a run for it.”
Measure Anything, Measure
Everything
Make it part of your culture
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects21
19. Making better
decisions from your
data
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
22
20. And allows us to manage scope
• Data supports our ability to prioritise features in a much more effective way
• It removes ambiguity of business benefits
• It supports continuous delivery of value
Data gives prioritisation context to requirements
23 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
21. So we need to run some tests
By generating some variants of your product you can simulate an experiment to start collecting data.
This can allow us to test hypothesis’ while not risking our core KPIs.
As long as you have measurement in place to identify the success or failure of a test.
Sometimes the data isn’t available
24 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
22. To chase value at every opportunity
Traditional projects
If we plan and execute our projects in this way, at what point do we know if what we delivered is the right thing?
Data + Learning = Pivoting
25 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects
Data driven projects
Continuous measurement (and analysis) allows us to continually chase the most valuable thing next, providing
organisational agility to change.
23. 29
Case Study
New Booking Proccess
Going Beyond ‘What Success Looks Like’
– Using Data to Achieve Successful
Projects
24. Our AS IS State
• An existing booking form on a
legacy technology stack that was
being deprecated.
• A change in Personal Card
Information (PCI) regulations
meant work needed to be
undertaken to remain compliant.
• An out-dated UI/UX that was no
longer in keeping with the rest of
the user’s journey.
• Was not fully supported across all
devices and had an adaptive
approach to mobile (2 code
bases).
The problem
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects30
29. Built the prototype
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects35
30. Available data
Funnel reports….
Reviewing our data
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects36
Conversion rates…
Missing data
31. Where are we and where do we want to be?
Defined our KPIs
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects37
The problem
• We had no mechanism within our
BI tool that gave us a baseline of
how breakfast sales performed.
The solution
• Implement a tracking mechanism
to collect data on our AS IS
breakfast sales performance.
• Work with stakeholders to define
a target for the next 6 months.
• Set thresholds that would alert to
higher or lower than expected
throughput.
Next
• Experiment…
32. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects38
35. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects41
36. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects42
OLD FORM NEW FORM
12 6Links Links
37. Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects43
OLD FORM NEW FORM
20 12Form elements Form elements
38. A better experience for all
User experience improvements
drove a conversion uplift
Benefits were delivered earlier
than planned
The data helped us pivot
• We never actually finished the
‘planned’ project..
• The ROI didn’t justify it.
We worked towards deprecating
2 legacy codebases
We now have a baseline of data and
measurements that we can build on
in the future
The benefits
Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects44
40. Wrapping it up…
1. Measuring success is linked right back to your initial requirements and
stakeholder desires. You’re helping your team prove that they’ve been met with
real data
2. Measuring success is a continuous process, measuring it at the end is too late…
make it part of your culture
3. Measuring success continuously reduces risk and allows for innovation and
agility
4. Evidence of success is the best reward of all, and will really help you as a BA
Conclusions
46 Going Beyond ‘What Success Looks Like’ – Using Data to Achieve Successful Projects