1. Copyright 2013 Freeform Dynamics Ltd 1
Big Data, Analytics and the
Future of Data Centres
Where are we and where are we going?
www.freeformdynamics.com
Tony Lock â Programme Director
tony@freeformdynamics.com
www.freeformdynamics.com
VMUG Meeting, Manchester February 12, 2013
2. Copyright 2013 Freeform Dynamics Ltd 2
About Freeform Dynamics
ï Industry analyst firm
ï Track IT industry developments and offerings
ï Track the evolution of IT related activity and needs in business
ï Advise both end user organisations and suppliers
ï Research approach
ï IT vendor and service provider briefings
ï Large scale studies - face to face, telephone and online
ï Community research programme
ï Investigate strategy, business case, architecture, best practice
ï Vendor patronage model allows free distribution
ï Media partnerships for both input and output
3. Copyright 2013 Freeform Dynamics Ltd 3
Agenda
ï Big data and Analytics
ï Where are we today?
ï The evolution of the data centre
ï Visions of the future
ï Will we ever reach Nirvana?
ï Closing thoughts
4. Copyright 2013 Freeform Dynamics Ltd 4
Big Data
ï What is it?
ï Is anyone doing it?
ï The only game in
town?
5. Copyright 2013 Freeform Dynamics Ltd 5
Defining Big Data?
ï Analogies
ï Panning for gold
ï Finding the needle in the hay stack
ï Identifying a weak signal in a very noisy environment
ï Find valuable patterns, trends, correlations, etc. in noisy,
unstructured, often complex, and high volume data sets
ï Doing analytics better / differently?
6. Copyright 2013 Freeform Dynamics Ltd 6
How much do you agree or disagree with the following
statements?
0% 20% 40% 60% 80% 100%
The emergence of advanced storage, access
and analytics solutions means the end of the
traditional RDBMS
Regardless of substance and reality of
emerging technologies and techniques, the
term âbig dataâ is currently being over-hypedâŠ
I have a clear understanding of what the term
âbig dataâ means
5-Totally agree 4 3 2 1-Totally disagree Unsure
7. Copyright 2013 Freeform Dynamics Ltd 7
The three Vs of Big Data
Volume
Variety
Velocity
Rule of thumb
Generally think
of Big Data
when two of
these three
apply
High physical volumes with low value
density
Different sources and formats or
information
Rapid rate of data movement, generation
or acquisition
8. Copyright 2013 Freeform Dynamics Ltd 8
The concept of value density
Traditional search and
document management
Traditional BI and data
warehousing
High value density
High value densityLow value density
Low value density
Structured
Unstructured
Structured
Unstructured
BIG
DATA
9. Copyright 2013 Freeform Dynamics Ltd 11
What level of growth are you seeing in the following
types of data within your organisation?
0% 20% 40% 60% 80% 100%
5 (Extremely high growth) 4 3 2 1 (No growth)
Structured data
(e.g. tabular data in RDBMSs)
Unstructured data
(e.g. documents, messages,
multimedia content, etc.)
10. Copyright 2013 Freeform Dynamics Ltd 12
In what form is your organisationâs most valuable or critical data
held (i.e. your crown jewels in information terms)?
0% 10% 20% 30% 40% 50%
Exclusively structured
Mostly structured
Equal split
Mostly unstructured
Exclusively unstructured
11. Copyright 2013 Freeform Dynamics Ltd 13
How is this changing?
0% 20% 40% 60% 80% 100%
Steady
shift
(25%)
Steady
shift
(21%)
No
change
(43%)
Rapid shift
towards value in
unstructured
data
(4%)
Rapid shift
towards value in
structured data
(7%)
12. Copyright 2013 Freeform Dynamics Ltd 14
To what degree does your organisation exploit its
information assets for analysis and decision making?
0% 20% 40% 60% 80% 100%
5 (Fully) 4 3 2 1 (Very poorly)
Structured data
(e.g. tabular data in RDBMSs)
Unstructured data
(e.g. documents, messages,
multimedia content, etc)
13. Copyright 2013 Freeform Dynamics Ltd 16
Use of traditional and emerging technologies
0% 20% 40% 60% 80% 100%
Legacy databases and file systems
General purpose RDBMS servers
High performance RDBMS configurations
OLAP multi-dimensional database systems
Write once read many (WORM) databases
Rule-based stream processing engines
In memory databases
Scale-out storage architectures
Distributed indexing and search
Distributed data analytics engines
5 (Extensive use) 4 3 2 1 (Not used at all) Unsure
-60% -40% -20% 0% 20% 40% 60%
Less use More use
Current level of use Change over next 3 years
14. Copyright 2013 Freeform Dynamics Ltd 17
How much do you agree or disagree with the following
statements?
0% 20% 40% 60% 80% 100%
Developments in advanced storage, access and analytics are allowing
us to tackle problems today that were either too hard or too expensive
to deal with in the past
Developments in advanced storage, access and analytics are allowing
us to take different and better approaches to tackling some key
business requirements
Vendors and consulting firms are well geared up to providing us with
the support and services we need to take advanced storage, access
and analytics on board effectively
We have a clear idea of the business benefits available to us through
the use of big data technologies and solutions
We have a clear idea of the advanced data storage and big data
analytic technologies that are becoming available
Database vendors are well geared up to support their customers with
appropriate licensing and commercial arrangements as data related
needs continue to evolve and become more demanding
5-Totally agree 4 3 2 1-Totally disagree Unsure
15. Copyright 2013 Freeform Dynamics Ltd 18
Elephants in the room
ï Access to data â what data is there and where is it?
ï Are there governance / regulatory / legal restrictions in play
concerning certain data sets?
ï Skill shortage?
ï IT skills
ï Numerical skills in user base
ï Just what questions could âBig Dataâ help with?
ï How do we exploit any results we generate?
ï Feedback into âmainstream systemsâ
16. Copyright 2013 Freeform Dynamics Ltd 19
Data Centre Evolution
ï Where are we now?
ï âPerfect Visionsâ
ï Will we ever get to Nirvana?
18. Copyright 2013 Freeform Dynamics Ltd 21
How much have you virtualised the following elements of your
IT landscape?
0% 20% 40% 60% 80% 100%
Your x86 server estate
Storage infrastructure
Your corporate network
Your desktop environment
Totally Extensively Partially A bit Not at all Unsure
Enterprise 481 respondents
19. Copyright 2013 Freeform Dynamics Ltd 22
How much have you architected your IT infrastructure in the
form of shared resource pools such a private clouds?
Unsure
6%
Not at all
40%
A bit
19%
Partially
20%
Extensively
11%
Totally
4%
Enterprise 481 respondents
20. Copyright 2013 Freeform Dynamics Ltd 23
YOUR VISION FOR DATA CENTRE COMPUTING
Following charts based on this question:
Putting all of the existing constraints and the current
state of the industry to one side for a minute, how
desirable would the following be as part of your perfect
IT vision?
21. Copyright 2013 Freeform Dynamics Ltd 24
0% 20% 40% 60% 80% 100%
All/most of our IT
requirements would be
fulfilled via hosted cloud
services
All/most of our own IT
infrastructure would be
based on private clouds
Highly desirable-5 4 3 2 Not at all desirable-1 Unsure or N/A
THE CLOUD HOSTING THING (Perfect IT Vision)
Enterprise 481 respondents
22. Copyright 2013 Freeform Dynamics Ltd 25
THE WHOLE CLOUD THING
0% 20% 40% 60% 80% 100%
All/most of your IT requirements are fulfilled via
hosted cloud services
All/most of your in-house (or co-located) IT
infrastructure is based on private cloud
architecture
Already there Within 1 yr Within 3 yrs Within 5 yrs
Within 10 yrs Later Never Don't know
Enterprise 481 respondents
23. Copyright 2013 Freeform Dynamics Ltd 26
0% 20% 40% 60% 80% 100%
Weâd be able to provision workloads and new systems capacity on our private
clouds with a few clicks on a management console
Automation would mean that shared resource usage was continually
optimised as demands fluctuate, with no human intervention
Weâd be able to migrate applications and workloads back and forth between
public and private clouds with ease
Hybrid cloud management capability would allow us to take an agnostic view
of resources, mixing and matching internal and external capacity freely
We would have end-to-end visibility across on-premise and hosted systems
for management and troubleshooting purposes
We would have a consistent/joined-up way of managing security and access
across in-house and hosted systems
We would have a consistent/joined-up way of managing and protecting data
across in-house and hosted systems
Weâd be taking a unified approach to operations and management across
servers, storage, networking, security, etc
Highly desirable-5 4 3 2 Not at all desirable-1 Unsure or N/A
OPERATIONS & MANAGEMENT (Perfect IT Vision)
Enterprise 481 respondents
24. Copyright 2013 Freeform Dynamics Ltd 27
OPERATIONS AND MANAGEMENT
0% 20% 40% 60% 80% 100%
New workloads and systems capacity are generally provisioned with just
a few clicks on a console
Use of key shared resources is continually optimised as demands
fluctuate, with no human intervention
You can quickly and easily move workloads back and forth between
public and private clouds
An agnostic view of resourcing means choices between internal and
external deployment are made purely on requirements and fitness for
purpose
You have end-to-end operational visibility across on-premise and hosted
systems
You have a consistent/joined-up way of managing security across in-
house and hosted systems
You have a consistent/joined-up way of managing data across in-house
and hosted systems
Already there Within 1 yr Within 3 yrs Within 5 yrs Within 10 yrs Later Never Don't know
Enterprise 481 respondents
25. Copyright 2013 Freeform Dynamics Ltd 28
0% 20% 40% 60% 80% 100%
Server, storage, networking, security and other specialists would
be working together seamlessly as part of a fully integrated ops
team
Other IT teams (developers, testers, support staff, departmental
IT, etc) would have self-service provisioning capability to obtain IT
resources
Non-technical users would have self-service provisioning
capability for new applications and services
IT activity and investment would revolve around the concept of
business services rather than systems
We would be able to easily and accurately charge or report IT
costs back to the business based on activity or consumption
Highly desirable-5 4 3 2 Not at all desirable-1 Unsure or N/A
ORGANISATION & SERVICES (Perfect IT Vision)
Enterprise 481 respondents
26. Copyright 2013 Freeform Dynamics Ltd 29
ORGANISATION AND SERVICES
0% 20% 40% 60% 80% 100%
Self-service provisioning is in place for IT teams
Self-service provisioning is in place for end users
IT activity and investment revolves around the concept of
business services rather than systems
You can easily and accurately charge or report IT costs back
to the business based on consumption
Already there Within 1 yr Within 3 yrs Within 5 yrs
Within 10 yrs Later Never Don't know
Enterprise 481 respondents
27. Copyright 2013 Freeform Dynamics Ltd 32
How much are the following standing in the way of progressing
towards the vision?
0% 20% 40% 60% 80% 100%
Inability of suppliers to deliver on visions and promises
Lack of interest/appreciation from senior management
The business not ready to upset the status quo
IT not ready to upset the status quo
Cultural impediments to investment in shared infrastructure
Ingrained IT funding models donât support new ways of working
Historical under-investment means the mountainâs too high to
climb
Lack of a formally defined vision and strategy
We simply donât have the time, resources or budget to focus on
anything other than short term priorities
Big impediment Significant challenge Minor challenge Not a problem Unsure
Enterprise 481 respondents
28. Copyright 2013 Freeform Dynamics Ltd 33
When it comes to driving forwards, how is it playing out, or
how do you think itâs going to play out in your organisation?
Other
4%
Things will be left largely
as they are for the
foreseeable future
18%
Just let adoption of modern
architectures, tools and
techniques creep along in
an ad hoc manner
20%
Build a modern
environment for new stuff,
and migrate older systems
into it incrementally
43%
Build a modern
environment for new stuff,
and leave older systems
where they are
11%
Single big transformational
initiative to modernise
things across the board
4%
Enterprise 481 respondents
29. Copyright 2013 Freeform Dynamics Ltd 34
Summing Up
ï Can you have you cake
(OK Apple) and eat it?
30. Copyright 2013 Freeform Dynamics Ltd 36
Moving forwards
âą Understanding is needed
âą Communicate â often and in language business users can understand
âą Ensure governance processes are âbig data awareâ
Senior business
awareness raising
âą Proactive local involvement in planning/prioritisation
âą Donât forget about getting âbig dataâ derived results back into business use
âą Clear policies/discipline around data usage
Minimisation of ad
hoc adoption
âąMonitor usage patterns to spot trends early
âąDonât get hung up on transient fads and fashions
âąEmbrace, substitute or block more persistent activity
Identify and deal with
real business issues
âąMore of an orchestration approach to IT leadership
âąArchitect systems with hard core and flexible edges
âąVirtualise the edge to handle diversity and personal use
Facilitate flexibility via
core IT
31. Copyright 2013 Freeform Dynamics Ltd 37
Adoption attitudes and tactics
Resist new ideas and solutions
Allow new stuff to creep in passively
Ad hoc opportunistic adoption
Draw a line, target new apps only
Establish beachhead, then expand
Big bang migration of everything
Recommended by
most early movers
Recipe for
disappointment
Rarely affordable
in cost or risk
32. Copyright 2013 Freeform Dynamics Ltd 38
Available for download now:
Big Data and Analytics
Dazzling new solutions or irritating new hype?
Available from www.freeformdynamics.com
http://www.freeformdynamics.com/fullarticle.asp?aid=1590
A Vision for the Data Centre
Are you a Mover, Dreamer or Traditionalist?
Available from www.freeformdynamics.com
http://www.freeformdynamics.com/fullarticle.asp?aid=1604
Hinweis der Redaktion
MB to modify to tune to how she likes to define the BD problem
Emerging foundation for consistent pundit/vendor viewA bit contrived, but useful, and need to know because the 3Vs and becoming increasingly prominent
When you look at some of the information feeds, including structured (logs, M2M, etc) and unstructured (social, web), along with unstructured internal sources, you find a lot of them are very âdiluteâ when it comes to valueThis these are the areas put into the âtoo difficultâ or ânot cost effectiveâ to solve categoryThis is where parallel, distributed scale-out architectures â aka Big Data come inStress that it is not just about unstructured â e.g. one of the most common starting points for IT departments cutting their teeth is systems log file analysis
But not just about Hadoop. Players like EMC, HDS, HP, IBM, Oracle and others all have offerings in the Big Data arenaZooming out, here are some of the common categories (there are many ways of categorising, this is more of a stack view)Scale out storage can be used for large traditional warehouses as well as big dataDistributed analytics can run on proprietary grids as well as things like HadoopThe point being that no element of Big data is exclusiveIndeed there is nothing exclusive about big data at all â it will live alongside existing solutions (see next slide_
Has advantages that derivative insights are a lot more accessible when extracted from Big Data environment and placed in a traditional environment â skills sets/familiarity, toolset availability, well established integration and propagation mechanismsm,etc