MWD Advisors White paper: Unlocking the potential of Big Data

mwd
advisors

Unlocking the potential of Big Data
Helena Schwenk
A special report prepared for Actuate
March 2012
Big Data is one of the hottest trends in IT industry circles. Although overused as a buzzword it is
generally characterised by the large, varied and rapidly growing volume of information that often
remains untapped by existing BI and data warehousing systems. It’s data that comes in all shapes and
sizes emanating from sources as diverse as mobile phone, sensors, smart energy meters, e-commerce
and social media sites. Yet within all this data lies significant value – especially for those businesses
that successfully tap into it, exploit it and put it to work for better business effect. As an emerging
technology discipline it also brings its own set of challenges in terms of scarcity of skills and IT best
practices. But for those organisations that can overcome these obstacles there’s huge potential to
unlock its value as a way of enhancing productivity, driving efficiencies and growth, and creating a
sustainable competitive advantage.
This is a special report prepared independently for Actuate. For further information about MWD
Advisors’ research and advisory services please visit www.mwdadvisors.com.

MWD Advisors is a specialist IT advisory firm which provides practical, independent industry
insights that show how leaders create tangible business improvements from IT investments. We use
our significant industry experience, acknowledged expertise, and a flexible approach to advise
businesses on IT architecture, integration, management, organisation and culture.

www.mwdadvisors.com

© MWD Advisors 2012

Unlocking the potential of Big Data 2

Summary
Defining Big Data is not Pinning down a definition for Big Data is an ongoing challenge
straightforward especially since the industry and marketplace has yet to reach
consensus. Until there is some form of agreement it’s best to
consider Big Data by its core characteristics. To begin with,
Big Data is not just about data volume – it also needs to take
into account its shape, speed, complexity and variety.
Secondly, Big Data can often be characterised as data that’s
either too difficult or not economically viable to store and
process using traditional data warehousing systems and BI
tools.

Analytics brings Big Data to life While it’s easy to get hung up on the complexities of storing
and unlocks its potential and crunching Big Data, this activity on its own will not help
you unlock its true business value. Leveraging advanced
analytic capabilities such data mining and text analytics, on
the other hand, can provide the means to enable you to
answer new questions, discover hidden insights, or find
unknown relationships in data to drive real business
advantage. In turn this can enable you to keep ahead of the
curve, discover new revenue streams, reduce costs, enhance
the customer experience and build sustainable competitive
advantage.

Harnessing and exploiting Big Data The mining of Big Data has the potential to reveal actionable
can bring significant rewards and valuable insights across multiple industries, organisational
sizes and business functions. This is possible not least
because at the same time as the quantity and variety of data
continues to grow, the technology for capturing, managing
and analysing all of this data is steadily improving – and at an
increasingly affordable price. So although we’re at an early
stage of market maturity, the potential for Big Data
applications to create value, enhance competitiveness and
improve productivity are widespread – including those for
better fraud detection, deeper levels of customer
segmentation and more accurate consumer behaviour
predictions.

Success requires blending business As you plan your Big Data initiative there are a range of
needs with investments in Big Data considerations that need to be taken into account to ensure
technology, data integration success. These include understanding the business need or
policies, and the right analytic challenge; securing the right level of commitment and
talent. investment from senior management; getting to grips with
the types and complexity of data sources at your disposal;
ensuring you navigate the technology landscape and choose
the right tools; and ensuring you invest in the right people
with the right skills to exploit your Big Data to its full effect.



Big Data makes its Big Impact
What’s the Big Deal with Big Data?
Unless you’ve been living in a vacuum it’s hard to avoid a conversation in today's business technology
circles without touching on the subject of Big Data. Similarly most press coverage of the topic centres
on Big Data as the new do-or-die technology that businesses need to leverage if they want to stay in
the game and remain one step ahead of the competition. It’s not surprising given these headlines
therefore that certain commentators have already written off Big Data as a bubble that is set to burst
and leave many IT organisations despondent in its wake.
While there is no shortage of hype – and there may very well be casualties along the way – Big Data’s
prominence and ascendency is driven by a very real business challenge, namely the unprecedented
growth of digital data across nearly every industry, region and size of organisation. It’s a challenge that
isn’t going to go away, as the figures demonstrate. According to a McKinsey Global Institute report1,
in 2010 enterprises globally stored more than 7 exabytes of new data on disk drives, while consumers
stored more than 6 exabytes of new data on devices such as PCs and notebooks. Likewise other
industry insiders point to the fact that that 90% of the data in the world today has been created in the
last two years alone. This tsunami of digital data is being generated by businesses and consumers alike
through social networks, sensors, online videos, e-commerce sites, GPS signals, printer streams and
Call Detail Records (CDR), to name but a few.
However, storing and managing this data is only one part of the challenge; the exponential growth in
information is also being matched by a strategic need and desire by businesses to find hidden nuggets
of information within this data. Extracting value and insight can help organisations keep ahead of the
curve in their quest to discover new revenue streams, reduce costs, enhance the customer
experience and build sustainable competitive advantage. Harnessing and extracting value from Big
Data is seen by many as a route to achieving these aims, where data is no longer purely seen as a ‘by
product’ of doing business, but is instead seen as an important asset that can be utilised to inform,
guide and improve the quality and speed of decision making.
In truth, any Big Data effort is likely to bring both opportunities and challenges for organisations. To
begin with, the management of Big Data is a difficult and complex undertaking. This is not only
because of the sheer volume of data that is being created, but also due to the variety of data types it
encompasses (such as unstructured and structured data), as well as the speed of its delivery, which in
some cases might be in real time. Similarly, once this data has been captured, stored and analysed,
organisations need to understand how those insights pertain to their business and how they can act
on them in a timely and effective manner. Yet in spite of this, the overriding fact remains that Big
Data, if used and harnessed successfully, can bring enormous benefit and value to organisations –
something that far outweighs the challenges and obstacles present in storing and processing it. In fact,
for some the benefits will only be limited by their ability to use data in more imaginative and valuable
ways.

What’s in a name?
Given its ubiquity as a term, coming up with a definition for Big Data is not as straightforward as you
might think, especially as the technology industry has yet to reach any kind of consensus. While
pinning down a definition can be akin to hitting a moving target, it’s helpful to consider Big Data by the
characteristics and traits it exhibits and in terms of how it differs from other more traditional data
management approaches.

1 Big data: The next frontier for innovation, competition, and productivity, May 2011, McKinsey Global Institute -
http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation



Our research suggests that to understand the full scope of Big Data management, it needs to be
framed in the following contexts:

 Big Data is not just about data volume – it also needs to take into account its shape, complexity
and scope. In contrast to more traditional data management approaches, it encompasses semi-
and unstructured data as well as structured data, and includes data generated not only by humans
but machines too.

 Similarly, the management of Big Data needs to takes into account data that is both ‘at rest’ –
where data is captured and analysed at a point in time – as well as ‘in motion’ – where data is
analysed as a continuous stream on the move.

 Big Data can often be characterised as data that’s not economically viable to store and process
using traditional data warehousing systems and BI tools. In this sense it often requires a new
technology, analysis and architectural approach to data management to harness it effectively.

 Don’t get distracted by size. Big Data is a subjective measure and can start from anything from
hundreds of terabytes to datasets that hit the petabyte range. What’s more important is the
context of its use in a traditional enterprise setting: Big Data projects typically apply to scenarios
where data has previously been too challenging to store and process or where data simply hasn’t
been accessible before.

 Big Data can be sourced from both inside and outside the organisation, whether it’s in social
media data streams, sensor logs or transactional data stored behind the firewall.



Business opportunities associated with Big Data
Tapping into the gold mine of Big Data
While a lot of buzz has focused on the technicalities of storing and processing Big Data this view often
overlooks the most important question: why should you care? The answer lies in uncovering the Big
Data sources that hold potential treasure troves of information that can be explored, mined and
combined with existing data to unlock secrets, opportunities and potential successes that are aligned
with the needs of your particular business. This means that there’s no simple ‘one size fits all’ answer.
The effective management of Big Data promises deeper and richer insights based on the ability to
work with individual records, rather than basing insights on an aggregated data slice (typically found
within a data warehouse), or a sample of the information to hand. This is especially true in
exploratory data analysis where analysts don’t always have a clear understanding of the questions they
want to ask of data. Without the benefit of Big Data technologies and techniques, analysts have no
choice but to work with partial data, which can introduce errors or limit the scope of analysis,
whereas analysing a complete set of data allows organisation to get answers to questions that haven’t
been posed before. In this sense, taking advantage of a Big Data opportunity requires a more creative
and inquisitive approach to data analysis and problem solving – one that combines the ‘science’ of
analytics and data discovery with the ‘art’ of applying it to real-world scenarios and revenue models.
Likewise, since a lot of what commentators call Big Data emanates from embedded sensors found in
mobile phones, medical devices, automobiles or smart energy meters, the use cases for its analysis can
extend to areas outside of the traditional domain of BI and analytics, within industries such as
healthcare, oil and gas, and transportation. In these scenarios Big Data can enable organisations to use
advanced correlation techniques to identify potentially useful patterns that would otherwise remain
hidden in petabytes of data.

Analytics brings potential to Big Data
Given all this potential it’s worth underlining the fact that Big Data on its own cannot unlock business
value. Instead it’s the application of Big Data to real-world business scenarios that provides scope for
competitive advantage. As shown in figure 1, it’s about pulling data together, combining the right
technologies and tools, applying analytics and creating actionable insights that business managers can
use to make better, higher quality and quicker decisions.



Figure 1: The Big Data mix

Business need

Big Data
technologies
Analytic &
skills & architecture
techniques

Data
integration

Actionable insights
Source: MWD Advisors

Getting the right mix enables organisations to sift through, find and exploit new patterns and
relationships in the data in order to, for example, identify risks, anticipate and respond to changes in
market conditions, and predict customer behaviour, conditions and events in ways that previously
haven’t been possible before. Similarly it can be used to add insight to existing analytics such as fine-
tuning customer segments for more targeted marketing campaigns, crafting better marketing
strategies, devising more profitable pricing strategies, offering more sophisticated product
recommendations and helping organisations discover new products and services.
It’s clear from some early use cases that the power of Big Data can yield some impressive business
results. However, the challenge for most organisations comes not only from how you process,
explore and mine Big Data, but also from understanding how those insights are relevant to your
business and how you can act on them in a timely and effective manner. In the next section we will
move on to talk about some of the most common use cases for leveraging Big Data in this business
context.



Understanding the use cases for Big Data
The possibilities for tapping Big Data to reveal valuable insights seem almost limitless, particularly as
it’s a phenomenon that impacts multiple industry sectors, organisational sizes and business functions.
Just as the quantity and variety of data continues to expand, the technology for capturing, managing
and analysing all of this data is steadily improving at an increasingly affordable price, allowing more
businesses to leverage and exploit the potential of Big Data. Although it’s still early days in terms of
real-world use cases, there are signs that organisations are actively pursuing Big Data opportunities to
create value, enhance competitiveness and improve productivity. Today organisations are mining the
data they’re currently capturing and storing, although may not necessarily be exploiting to its full
potential. At present, our research suggests that the usage scenarios for Big Data fall into one of four
broad business opportunities and drivers, as shown in figure 2.

Figure 2: The business opportunity for Big Data

Business Driver Opportunity Example
Improving Identifying and Telcos can analyse growing volumes of CDRs, together with
operational preventing customer interaction usage, network and transactional data to discover and
efficiencies churn predict new forms of churn in their network.
Fraud detection Insurance companies can identify patterns of fraudulent behaviour
much much faster found in terabytes of online, mobile and
transactional data for insurance claim fraud and anti-money
laundering,
Pinpointing areas for cost Retailers can use data captured from loyalty reward programs, and
efficiencies in store, mobile and online transactions to optimise and improve
margins for product inventory, and markdowns
Mitigating risk Financial service companies can monitor and analyse financial data
streams in faster timescales to identify and minimise their credit
and market risk exposure.
Enhancing the Understanding customer Consumer Package Goods companies can acquire and mine
customer experience sentiment unstructured data from social networks to get an overall picture of
their brand’s perception and conduct real-time market research.
Fine tuning customer Financial institutions can segment customers by credit card
segmentation behaviour at a finer level of granularity to target and tailor
products more effectively to specific risk profiles
Gaining a 360-degree Organisations of all sizes can capture and accumulate a wider range
view of the customer of customer attributes to gain deeper and more accurate insight
into customer behaviour and model it with greater precision.
Improving revenue Identifying new sales Web-based companies can get a fuller picture of visitor usage and
generation opportunities purchase patterns to help optimise website design, content
creation and develop product recommendations that boost traffic
and sales.
More granular customer A retailer can collect and mine customer purchase data to micro-
targeting segment its customer base that is used to optimise its product mix,
pricing, and promotions more accurately
Driving strategic Better planning and Utilities and energy companies can tap into vast volumes of smart
change performance meter data to accurately predict retail demand and control supply
management costs in ways that have not been possible before.
Understanding new A credit card provider can create value from the wealth of data it is
markets storing and analysing by selling consumer insights based on the
data streams it generates from processing payments.
Discovering and Healthcare providers can aggregate and analyse enormous volumes
developing new products of clinical and claims data, to find the next big ‘super’ drug that will
or services help .

Source: MWD Advisors



Bigger, better and faster
While some of these application areas and use cases are familiar and well understood by BI and data
warehousing communities, what’s different now is the scale and scope of analysis that Big Data can
enable. In other words, if leveraged in meaningful and more accurate ways Big Data can help you
exploit information to do things bigger, better and faster. This in turn will place new requirements on
BI and analytic toolsets as they are called upon to support the volume, speed, variety and workload
demands of Big Data. It’s an effort that will require you to look seriously at the technologies used to
drive both your current and future data management strategies and information needs.
To begin with, your BI environment will need to extend its support for analytic techniques such as
data mining, predictive modelling, natural language processing, machine learning and advanced SQL, as
well as improving support for collaboration, data discovery and visualisation techniques to help
interpret the results of Big Data analysis.
At the same time this needs to be married from a data management and integration perspective with
capabilities for sourcing new forms of data, including unstructured and structured data, the ability to
support both high and low latency data demands, as well architectural support for scale-out and high
speed data processing.
Today these Big Data challenges cannot be solved by a single platform or engine but instead need to
employ a variety of technologies, components and architectures. These may include technologies such
as Hadoop, MapReduce and distributed NoSQL databases, but it could also include technologies such
as in-memory databases, columnar databases and massively parallel processing architectures. However
for some, the real potential value of Big Data will only come when it’s merged and integrated with
existing business processes and data assets, such as a data warehouse, to provide a fuller and more
complete picture of their business.
Finally, any Big Data effort will require you to think carefully about sourcing and investing in the right
people, analytic skills and experience to make sure you can take advantage of the huge opportunities
that Big Data presents.



Where to start on your Big Data journey
As you plan to embark on a Big Data initiative there are a range of considerations to take into
account and challenges to overcome if your initiative is to realise its full potential. You need to
develop a practice that involves assessing business priorities and needs and match these with
investments in Big Data technology and techniques, data integration policies and the right analytic
talent. To assist you on the path to Big Data success the following steps provide guidance about how
and where to start your Big Data journey.

 Get buy-in and commitment. It’s true to say that all IT programmes benefit from having
senior-level sponsorship and buy in, but this is especially true in the case of Big Data projects. A
sponsor needs not only to invest time and money in any effort but also match this with a
compelling vision and understanding of how Big Data can unlock real business potential for your
organisation.

 Choose your data sources. A large part of the Big Data effort involves assessing the type and
format of data sources you want to use. In many cases this could mean considering opportunities
for analysing new types of data such as log files, sensor data or video streams that were
previously not available or possible before.

 Good data preparation reaps rewards. It doesn’t make sense to always subject Big Data to
the same rigorous data cleansing, scrubbing and matching routines required in an enterprise data
warehousing environment. However, in certain scenarios you will still need to transform the data
and apply hygiene routines to Big Data in order to maximise its potential, for example by ensuring
you have prepared the data for analysis and rectified any data quality issues in the source data.

 Change the way you think about data. The ability to analyse all of your data rather than just
a subset or sample will require a subtle but different analytic mindset. Big Data environments are
often regarded as exploratory platforms where analysts can dig and play around in the data as
they attempt to uncover new and interesting insights. It’s a mindset that requires a more creative
and inquisitive approach to data analysis and problem solving, and one that combines traditional
analytic disciplines with the ability to apply these to real-world business scenarios.

 Pick your tools. With such an array of technologies and architectures to choose from, expect a
considerable part of any Big Data effort to be spent on understanding and navigating the
technology landscape. You need to consider key capabilities such as the performance, scale, and
data delivery rates of each tool or platform alongside support and integration with BI and
advanced analytic tool and techniques.

 Invest in skills, skills, skills. Finding the right talent to utilise Big Data technologies and
techniques will continue to be a challenge for most. Those of you who are new or have had
limited exposure to disciplines such as Hadoop, data mining or statistics will need invest time in
sourcing or training staff. However, this is only part of the story: there should also be an equally
concerted effort to employ and develop those skills for aligning the data with the business, so
insights derived from Big Data can be used to drive better decision-making and business
outcomes.


MWD Advisors White paper: Unlocking the potential of Big Data

Recommended

Recommended

More Related Content

More from Actuate Corporation

More from Actuate Corporation (20)

Recently uploaded

Recently uploaded (20)

MWD Advisors White paper: Unlocking the potential of Big Data