C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing

“The Cloud” – Impacts the Organization Structure
1. What is driving IT / Businesses to Cloud
2. Traditional IT Organization Impact
3. Traditional vs. Design-for-Fail, On-premise vs. Off-premise
4. Example Big Data / Cloud Storage Products and Directions
© 2013 IBM Corporation
Cloud Storage Briefing - December 3, 2013
Provided by: John Sing, Executive IT Consultant, San Jose, California singj@us.ibm.com

IBM Cloud Storage Briefing – December 3, 2013
2
What is driving IT and Businesses to Cloud

Time-to-Delivery
Competitive Advantage
Revenue
“Time is Money”
Localized, any time
any where
Dynamic (Elastic)
Modern 21st Century Cloud Business Value
3
Centralized

Value delivered
Storage Provisioning
Continuous Access to data
From traditional
Weeks
To cloud
Minutes
For users
Storage Capacity Fixed
Reduced storage admin
costs
Up to 50% savings
For IT
Reduced energy costs Up to 36%
Increased storage utilization From 50% Up to 90%

Primary drivers for move to cloud = business reasons
4
Competitive Advantage,
Revenue
http://www.kpmg.com/global/en/issuesandinsights/articlespublications/cloud-service-providers-survey/pages/service-providers.aspx

5
Bandwidth availability is tipping point for adoption of “The
Cloud”………
 Worldwide broadband bandwidth availability is
becoming commonplace
 Facilitates a pervasive web services delivery model
– (i.e. “The Cloud”)
 Hosted in mega data centers with massive amounts:
– Processors, Storage, Network
 Today, when above 3 come together in a geo:
–We are seeing small, medium on-premise data
centers worldwide rapidly disappearing, off-premise,
into the cloud
 The real question:
– Is traditional IT re-capturing / replacing workloads
when they move off-premise to Cloud ?

6
Cloud Mega Data Centers = new modular IT implementation style…
 Internet-scale centers…..
 Data:
–10s / 100s petabytes
 Servers:
–100,000s ….
Workloads:
–Require server clusters
of 100s, 1000s, 10,000,
more …..
Modular implementation

Amazon Web Services
Amazon Web Services 1Q12: 450,000 servers
7
Amazon Perdix Modular Datacenter
1Q12:
450,000
Servers
estimated
EC2 17K core, 240 teraflop cluster 42
nd fastest supercomputer in world
1Q13: >
2 trillion
objects in S3
1Q13: 1.1 M
req/sec
http://aws.typepad.com/aws/2012/04/amazon-s3-905-billion-objects-and-650000-requestssecond.html
http://gigaom.com/cloud/how-big-is-amazon-web-services-bigger-than-a-billion/
http://aws.typepad.com/aws/2013/04/amazon-s3-two-trillion-objects-11-million-requests-second.html

8
Growth of
The Cloud
by 2016
 Mobile
 Geo-locational
 Real-time data
 Shift to cloud
mega-data centers
Cisco
already
knows
> 50%
workload is
in the cloud
http://www.datacenterknowledge.com/archives/2012/10/23/cisco-releases-2nd-annual-global-cloud-index/
Source:
> 50% in
cloud

Cloud: No longer exploratory
9
•Cloud is at the end of its
beginning phase and has gotten
serious
•Private cloud is growing, but
giving way to hybrid cloud
•Service providers, VARs, SIs
are rising to the cloud opportunity
•Cloud adoption is strong across
large enterprise as well as SMB.
Expectations: Cloud computing
will be "just computing" by 2018

10
So, What is a Cloud, really?
Why does it impact Traditional On-Premise
IT organization so heavily?
Extracted from presentation: “Building a 21st Century Cloud Storage Service” by John Sing:
http://snjgsa.ibm.com/~singj/public/2013_Berlin_System_Storage_x_Pure_Symposium/sCS05_John_Sing_Building_21st_Century_Cloud_Storage_Service_Industry_Best_Practice.ppt

To users, cloud seems “easy”, “instant”, “self-service”.
So what has to happen in the background?
 Some would say that virtualization = cloud
 Some IT traditionalists would say that cloud
is nothing more than much better managed
centralized, automated data centers
 Unfortunately, such statements severely
undersize the essential organizational
element
 To provide true cloud services, you must
also execute a significant shift in:
11
– Organizational lines
– Processes
– Workflows
– Workload types
– Required skill sets Key message

This is the
cloud-enabled
data center
journey
1. Virtualized
2. Deployed
3. Optimized
4. Enhanced
5. Monetized
12
Cloud
adoption
maturity
levels
Level of cloud capability
(macropatterns)
http://www.redbooks.ibm.com/abstracts/redp4893.html
IBM
Redpaper

What’s most important: cloud macropattern workflows
3. Adv
IaaS
13
1. Simple IaaS
4. ITIL Managed
IaaS
2. Cloud
Mgmt

Tivoli Storage
Productivity Center
Problem! Traditional IT organization looks nothing like this workflow!
Cloud micro-pattern workflows
14
 Are you ready?
Smart Cloud
Storage Access
IBM Storwize V7000, SVC, XIV Tivoli Storage Manager

IBM Redpapers: Building Cloud Enabled Data Center / Service Provider
http://www.redbooks.ibm.com/abstracts/redp4893.html http://www.redbooks.ibm.com/abstracts/redp4873.html
15
http://www.redbooks.ibm.com/abstracts/redp4912.html

Example: IBM Storage products within the Cloud workflow
16
Non-Technical Users
Self Provisioning Requests for Windows or Linux
Server, Application Owners, Developers users, etc.
P9: IBM P9: IBM S SmmaartrCtClolouudd S Stotoraraggee A Acccceessss
PP88: : I BIBMM T Tivivoolil iS Stotoraraggee P Prorodduucctitvivitiyty C Ceennteterr
P0: IBM SVC / Storwize
V7000 U
OS and end user consumption
Ethernet Network
File
P0: IBM SONAS
Block
P0: IBM XIV
Virtualizes
IBM or 3rd party Storage
arrays(HP, NetApp, EMC, etc.)
CIFS / NFS
Provisioning Requests for LUNs to be
assign/consume by either to physical or Virtual
Servers
LUN
LUN
LUN
Physical or
Virtual
Servers
LUNs
LUN
eMail
DB2
SAP
ERPs
TPC/Storage Admin
16

Key Cloud organizational learning point:
17
Cloud involves major re-alignment of IT organization, skills
 Re-alignment of IT processes, to facilitate real-time, elastic management, monitoring,
delivery based on service catalog
– Aligned with the Lines of Business revenue generation / competitive advantage needs
(requires full-time liason positions)
 Creation of service catalog requires IT to invest different efforts into
design/automation of IT capability
– New, additional skill requirements, aligned along a very different organizational structure,
metrics, and speed criteria
 Provide governance that addresses risk of unauthorized or rogue access to services
– Only appropriate approvals and credentials, thus new emphasis on network + security
 Addressing resistance to change within IT organization is the biggest success factor
If the on-premise IT organizations is unable to change…..
– this is also a major off-premise cloud driver

This organizational shift is a main reason why “ready-to-go” cloud workflow
products (such as OpenStack) are so attractive:
OpenStack already
has all cloud workflows
ready for production
18
Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/

OpenStack is comprised of seven core projects that form a complete
Cloud Infrastructure as a Service (IaaS) solution
IaaS
19
Compute (Nova)
Block Storage (Cinder)
Network (Neutron)
Provision and manage
virtual resources

Dashboard (Horizon)
Self-service portal
Image (Glance)
Catalog and manage
server images
Identity (Keystone)
Unified authentication,
integrates with existing
systems
Object Storage (Swift)
petabytes of secure,
reliable object storage
Nova
Neutron
Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
IaaS
Understand OpenStack
to understand IBM
Cloud Storage
directions
Horizon
Glance Swift
Keystone
Cinder

20
Knowledge Check
Did you know: two different types of IT architectures have emerged
Design-for-Fail IT implementation has some similarities,
but clearly isn’t the same, as Traditional IT architecture

21
Today there are two major types of IT Cloud
architectures and workloads:
Transactional IT
“Systems of Record”
Internet Scale
Workloads
“Systems of Engagement”
Cloud, High Availability,
Resiliency, Disaster
Recovery
characteristics
Can be adapted to Cloud
“agnostic / after the fact”
Data Strategy Can leverage traditional
tools/concepts to understand /
implement cloud
Storage/server virtualization and
pooling
Automation End to end automation of server /
storage virtualization
Commonality Apply master vision and lessons
learned from internet scale data
centers

22
The other major type of IT Cloud architecture and
workload is:
Transactional IT
Internet Scale
Workloads
“Systems of Engagement”
Cloud, High Availability,
Resiliency, Disaster
Recovery
characteristics
Can be designed “Agnostic / after
the fact” using server or storage
virtualization, replication
Cloud capabilities are
“designed into software stack
from the beginning”
Data Strategy Use traditional tools/concepts to
understand / know data
Storage/server virtualization and
pooling
Proven Open Source toolset
used implement failure
tolerance and redundancy in
the application stack
Automation End to end automation of server /
storage virtualization and replication
End to end automation of the
application software stack
providing failure tolerance
Commonality Apply master vision and lessons
learned from internet scale data
centers
Apply master vision and
lessons learned from internet
scale data centers

Transactional IT Internet scale wkloads
Today: two different types of IT
23
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/

Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/ Transactional IT Internet scale wkloads
Today’s two major IT workload types
24

How to build these two different IT architectures
25
Transactional IT
Internet scale wkloads

26
What You (Consumer) Get with These different
approaches:
Transactional IT

Policy-based Clouds and Design-for-Fail Clouds are
workload optimized architectural choices
27
 Policy-based Clouds
• Purpose optimized for longer-lived virtual
machines managed by Server Administrator
• Centralizes enterprise server virtualization
administration tasks
• High degree of flexibility designed to
accommodate virtualization all workloads
• Significant focus on managing availability and
QoS for long-lived workloads with level of
isolation
• Characteristics derived from exploiting enterprise
class hardware
• Legacy applications
 Design-for-fail Clouds
• Purpose optimized for shorter-term virtual
machines managed via end-user or automated
process
• Decentralized control, embraces eventual
consistency, focus on making “good enough”
decisions
• High degree of standardization
• Significant focus on ensuring availability of
control plane
• Characteristics driven by software
• New applications
Transactional IT Internet scale wkloads

Example: Traditional IT vs. Hadoop for Big Data
Traditional approach : Move data to program
Big Data approach: Move function/programs to data
28
Database
server
Data
Query Data
return Data
process Data
Master
node
Data
nodes
Data
nodes
Data
Application
server
User request
Send result
User request
Send Function to
process on Data
Query &
process Data
Data
nodes
Data
nodes
Data
Data
Send Consolidate result Data
Traditional approach
Application server and Database
server are separate
Analysis Program can run on
multiple Application servers
Network is still in the middle
Data has to go through network
Designed to analyze TBs of data
•Big Data Approach
 Analysis Program runs where the
data is : on Data Node
Only Analysis Program has to go
through the network
Analysis Program is executed on
every DataNode
Designed to analyze PBs of data
Highly Scalable :
1000s Nodes
Petabytes and more
Thank you to: Pascal VEZOLLE/France/IBM@IBMFR and Francois Gibello/France/IBM for the use of this slide

Example: Traditional IT vs. Hadoop for Big Data
2299
Database
server
Data
Query Data
return Data
Application
server
process Data
User request
Send result
Master
node
Data
nodes
Data
nodes
Data
User request
Send Function to
process on Data
Query &
process Data
Data
nodes
Data
nodes
Data
Data
Send Consolidate result Data
Example: How many hours of Clint
Eastwood appears in all the movies he
has done?
Task: All movies need to be
parsed to find Clint’s face
•Traditional approach :
1)Upload a movie to the application server
through the network
2) The Analysis Program compares Clint’s
picture with every frame of the loaded movie.
3) Repeat the 2 previous steps for every movie
•Big Data Approach :
1)Send the Analysis Program and Clint’s
picture to all the DataNodes.
2) The Analysis Program in every DataNode
(all in parallel) compares the Clint’s picture
with every frame of the loaded movie.
3) The results of every DataNodes are
consolidated. A unique result is generated.
Traditional approach : Move data to program
Big Data approach: Move function/programs to data
Thank you to: Pascal VEZOLLE/France/IBM@IBMFR and
Francois Gibello/France/IBM for the use of this slide
Note: Hadoop typically uses direct attached storage

Hadoop principles: Storage, HDFS and MapReduce
 Hadoop Distributed File System = HDFS : where Hadoop stores the data
public static class TokenizerMapper
public static class TokenizerMapper
extends Mapper<Object,Text,Text,IntWritable> {
private final static IntWritable
extends Mapper<Object,Text,Text,IntWritable> {
private final static IntWritable
one = new IntWritable(1);
private Text word = new Text();
public void map(Object key, Text val, Context
StringTokenizer itr =
private Text word = new Text();
public void map(Object key, Text val, Context
StringTokenizer itr =
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
context.write(word, one);
}
}
}
public static class IntSumReducer
extends Reducer<Text,IntWritable,Text,IntWrita
private IntWritable result = new IntWritable();
public void reduce(Text key,
30
– HDFS file system spans all the nodes in a cluster with locality awareness
 Hadoop data storage, computation model
– Data stored in a distributed file system, spanning many inexpensive computers
– Send function/program to the data nodes
– i.e. distribute application to compute resources where the data is stored
– Scalable to thousands of nodes and petabytes of data
one = new IntWritable(1);
new StringTokenizer(val.toString());
Iterable<IntWritable> val, Context context){
int sum = 0;
for (IntWritable v : val) {
sum += v.get();
MapReduce Application
1. Map Phase
(break job into small parts)
2. Shuffle
(transfer interim output
for final processing)
3. Reduce Phase
(boil all output down to
a single result set)
Shuffle
Return Result Set a single result set
. . .
new StringTokenizer(val.toString());
while (itr.hasMoreTokens()) {
word.set(itr.nextToken());
context.write(word, one);
}
}
}
public static class IntSumReducer
extends Reducer<Text,IntWritable,Text,IntWrita
private IntWritable result = new IntWritable();
public void reduce(Text key,
Iterable<IntWritable> val, Context context){
int sum = 0;
for (IntWritable v : val) {
sum += v.get();
. . .
Distribute map
tasks to cluster
Hadoop Data Nodes
Data is loaded,
spread, resident
in Hadoop cluster
Performance =
tuning Map Reduce workflow,
network, application,
servers, and storage
http://www.ibm.com/developerworks/data/library/techarticle/dm-1209hadoopbigdata/
http://blog.cloudera.com/blog/2009/12/7-tips-for-improving-mapreduce-performance/
http://www.slideshare.net/allenwittenauer/2012-lihadoopperf

Transactional IT
Two different types of cloud tooling
Cloud storage tooling will most likely reside:
 In the external shared storage stack for policy-based traditional transactional IT:
31
– External IBM Smarter Storage hardware and software for block and file storage
 In the virtualized server, direct attach storage, application stack for design-for-fail:
– IBM SmartCloud software, IBM participation in Open Stack, IBM Softlayer
 Both are appropriate, match to proper environment
http://www.slideshare.net/johnsing1/s-bd03-infinitybeyond2internetscaleworkloadsdatacenterdesignv6speaker

Read all about it. Google published this information into the public domain in
2009. 2nd Edition of this book published July 2013(includes Flash storage)
32
 By Google:
– Luiz Andre Barroso
– Uri Holze
 Available to all, free of
charge
Download original edition at: http://www.morganclaypool.com/doi/pdf/10.2200/S00193ED1V01Y200905CAC006
New! 2nd Edition published July 2013: http://www.morganclaypool.com/doi/abs/10.2200/S00516ED2V01Y201306CAC024
Video of Luis giving one of these lectures: http://inst-tech.engin.umich.edu/leccap/view/cse-dls-08/4903
http://www.barroso.org/

33
Size of Cloud Market:
Magnitude of On-premise vs. Off-premise

Size of Server, Storage, Networking aggregate marketplaces
34
2013 2017
$104B $117B
Compound Growth Rate 2013-2017
Cloud Service Provider (CSP) 25%
Enterprise Private Cloud (EPC) 23%
Non-Cloud -7%
Total 3%
Source: IBM
37% is for Storage

Off-premise is
clearly the growth
Cloud
Services
Cloud adoption continues acceleration through 2017
35
On premise vs. off premise spend
September 2013
CSP, $33B
25% CGR
EPC, $24B
23% CGR
Source: IBM
Enterprise
On-premise
Non-Cloud
Cloud IaaS
Cloud server,
storage,
networking
$57B, 24%CGR
48% of Total
Non-Cloud
$60B,
-7%CGR
52% of Total
Off
premise
On
premise
area

36
IBM Big Data / Analytics Storage Positioning

We are building real-time, integrated stream computing on massive scale
37
n d
Chart in public domain: IEEE Massive File Storage presentation, author: Bill Kramer, NCSA: http://storageconference.org/2010/Presentations/MSST/1.Kramer.pdf

Intelligence
Analysis,
However, note there are multiple types of Big Data
38
Data in
Motion
Data at
Rest
Data in
Many Forms
Real-time Analytics
Streams
Information
Ingestion and
Operational
Information
Decision
Management
BI and Predictive
Analytics
Navigation
and Discovery
Video/Audio
Network/Sensor
Entity Analytics
Predictive
Landing Area,
Analytics Zone, Archive
Raw Data
Structured Data
Text Analytics
Data Mining
Entity Analytics
Machine Learning
Exploration,
Integrated Warehouse,
and Mart Zones
Discovery
Deep Reflection
Operational
Stream Processing Predictive
Data Integration
Master Data
Batch parallel Big
Data processing
Real-Time
In-memory servers
Data Warehouse
Traditional IT
IInnffoorrmmaattiioonn GGoovveerrnnaannccee,, SSeeccuurriittyy aanndd BBuussiinneessss CCoonnttiinnuuiittyy

Intelligence
Analysis
Example: IBM end to end Big Data portfolio
39
Data in
Motion
Data at
Rest
Data in
Many Forms
Real-time Analytics
Streams
Information
Ingestion and
Operational
Information
Decision
Management
BI and Predictive
Analytics
Navigation
and Discovery
Video/Audio
Network/Sensor
Entity Analytics
Predictive
Landing Area, Analytics
Zone and Archive
Raw Data
Structured Data
Text Analytics
Data Mining
Entity Analytics
Machine Learning
Exploration,
Integrated Warehouse,
and Mart Zones
Discovery
Deep Reflection
Operational
Predictive Stream Processing
Data Integration
Master Data
IBM BigInsights
IBM
InfoSphere
Streams
IBM Data Warehouse
products
IInnffoorrmmaattiioonn GGoovveerrnnaannccee,, SSeeccuurriittyy aanndd BBuussiinneessss CCoonnttiinnuuiittyy
IBM STG: x, p, PureSystems,
Platform Computing
IBM STG: x, p,
PureSystems, Platform
Computing
IBM SWG

Customer disk GB cost expectation
Optimized Multi-Temperature Data
Optimized Multi-Temperature Data
Warehouse
Warehouse
(USA): 30 to 70 cents/GB
oAll Flash
oAll Flash
– FlashSystem
– FlashSystem
oHybrid
oHybrid
– DS8000 EasyTier
– Storwize EasyTier
– FlashSystem Solution (VSC +
– DS8000 EasyTier
– Storwize EasyTier
– FlashSystem Solution (VSC +
FlashSystem)
FlashSystem)
– XIV
– XIV
oPureSystems
oPureSystems
– PureFlex (Storwize w/EasyTier)
– PureData for Transactions (Storwize)
– PureData for Analytics (Netezza)
– PureFlex (Storwize w/EasyTier)
– PureData for Transactions (Storwize)
– PureData for Analytics (Netezza)
– IBM Big Data Networked Storage
– IBM PureData System for Hadoop
with pre-installed IBM BigInsights
– Generally Available September 2013
Example: IBM Big Data Storage positioning
40
Hadoop
Hadoop
oStorage for Hadoop
oStorage for Hadoop
– IBM Big Data Networked Storage
Solution for Hadoop
Solution for Hadoop
oPureSystems
oPureSystems
– IBM PureData System for Hadoop
with pre-installed IBM BigInsights
– Generally Available September 2013
Customer disk GB cost expectation
(USA): 10 to 15 cents/GB with
direct or SAS attach, extreme density

41
Cloud Storage Directions

Data Growth Types in the Cloud
42
BLOCK
FILE
OBJECT
Worldwide File-based vs Block-based
Storage Capacity Shipments 2008-2015
Object
File
Block
 Block – Traditional data is structured and managed by OS i.e. Database
 File – High growth data is unstructured and managed by OS i.e. File System
 Object – Higher growth data is unstructured and managed by Application

Object Storage – fundamental type of storage for Cloud
4433
Object Storage
Network “Best Case” delivery
Best usage = data that doesn’t
change
i.e. backups, archives, digital images,
virtual machine images….
Distance limited only to
acceptable network latency
Servers
Applications
 Object storage features are minimal compared to NAS or SAN:
– store, retrieve, copy, delete files
– control which users can do what
 Protocol usually HTTP interface Object Storage API (RESTful API)
– Can be in URL format for WWW access
 Application is responsible for tracking object unique IDs and supplying
that unique ID to retrieve data from object storage
 Typically longer response times than either NAS or SAN
– Slower throughput compared traditional file system means object storage
unsuitable for data that changes frequently
 Typical usages: great fit for data that doesn't change much:
– backups, archives, video and audio, VM images
– i.e. internet-scale repositories of data
– This is why it is so essential to Cloud
No concept of file system. Rather, application saves object (files + additional metadata) to the object store via PUT API cmd,
application gets a unique keyfor the saved file, application must provide that unique key to a GET API command to
retrieve files
Can imbed searchable
metadata directly into
object storage system

Objects are a natural fit to “born on cloud” data (mobile, social)
 Objects are written once and never modified (although they can be replaced)
– this describes most born on the cloud data
Consumer Apps Business Apps
44
– Pictures, e-mails, movies, tweets, blog-posts, web pages, etc.
– This data is both consumer and enterprise
– Much of this data is accessed from mobile devices
 Hence Object Storage is essential to participate in Cloud Storage world
Pictures Collaboration Backup Archive
Rackspace

Object Storage
Object
APPLICATION
IIPP NNeettwwoorrkk
OObbjjeecctt AAPPII
OBJECT
CONTAINER
Object API
Object API
Object I/O
Block I/O
Storage: SAN / NAS / Object
4455
NAS
(Network Attached Storage)
AAPPPPLLIICCAATTIIOONN
File I/O
IIPP NNeettwwoorrkk
File I/O
FFIILLEE SSYYSSTTEEMM
SSTTOORRAAGGEE
Block I/O
CIFS, NFS, HTTP
SAN
(Storage Area Network)
AAPPPPLLIICCAATTIIOONN
File I/O
FFIILLEE SSYYSSTTEEMM
Fibre Channel SAN
or iSCSI
SSTTOORRAAGGEE
Block I/O
FICON, FC, iSCSI, FCoE
SSTTOORRAAGGEE
Object Storage (HTTP)
Block I/O

IBM Cloud Storage – current products and future directions
Traditional IT:
 IBM Smart Cloud Storage Access - to provide P9 and P8 Self-Service Automation (storage)
 IBM Tivoli Storage Productivity Center – to provide P6 Storage Virtualization Management
 IBM Storwize Family and XIV – provide P0 storage virtualization including enterprise best-in-class
46
OpenStack exploitation
 IBM SONAS and V7000 Unified - provide P0 storage virtualization for file storage
Cloud Storage and Object Storage Directions:
 Exploitation of OpenStack Cinder for block storage
 Exploitation of OpenStack Swift for software-defined object storage approach
 Best-in-class OpenStack enterprise exploitation
 Design for Fail / Cloud Native / Internet scale IT :
 Exploit SoftLayer for Cloud Native
 Migrate IBM SmartCloud workloads into Softlayer workflow approach over time

OpenStack components; IBM Storage strategic exploitation
47
Horizon
Nova
Cinder
Swift
Neutron
Glance
Keystone
New in Havana
Metering (Ceilometer)
Basic Cloud Orchestration &
Service Definition (Heat)
Oslo
Shared Services
Software
Defined
Object
IBM
Storage
SVC / Storwize
XIV
Future
directions

OpenStack Object Storage component – “Swift”
 An open source, highly available, distributed, eventually consistent object store
48
– Two tier architecture consisting of client facing proxies and storage servers
– Information protected through three-way replication (by default)
– Supports geo-distribution
– The dominant design for scale-out object stores
 Swift was developed as pure software
disconnected from
hardware
– Typically implemented on
storage rich servers, e.g.,
– IBM x3630 M4
 Swift in production at Softlayer,
Rackspace, Korea
Telecom, Wikimedia,
 UCSD, Internap,
Sonian, MercadoLibre, . . .
Internet
or
Intranet
Internet
or
Intranet
Private
Network
Clients send
REST
requests
Storage Servers (account,
container and object) store, serve
and manage data and metadata
partitioned based upon ring
Proxy Layer (public face)
authenticates and forwards
to appropriate storage
server(s) using ring

IBM Object Storage Cloud and IBM OpenStack directions
 2014 directions: a pure IBM Storage Software offering, based on OpenStack Swift,
with IBM value-add, providing object storage interface with highly available, cost
effective, scale out storage features.
http://<host>/<api versions>/<account>/<container>/<object>
…
49
– Leverage open source assets for a lightweight and flexible, interoperable foundation
 Target Markets
– Telco/CSP, MSP, HealthCare, FSS
 Scope
– Simple and Easy to use management
• Ease of Use XIV/Storwize GUI
• Build on community tools
• Smart Swift infrastructure management
• Cloud Support: Provisioning, Metering
– Multi-tenant security
• Authentication and management isolation
– Compliance
• Object Retention
– Architecturally able to scale
• To thousands of nodes
• Initial offerings much smaller
Object URL call:
…
Private Network
…
Zone 1 Zone 2 Zone n

IBM SmartCloud capabilities for major IT architectures
50
Cloud Enabled
Scalable
Virtualized
Automated Lifecycle
Heterogeneous Infrastructure
Cloud Native
Elastic
Multi-tenant
Integrated Lifecycle
Standardized Infrastructure
+
Existing
Middleware
Workloads
Emerging
Platform
Workloads
Compatibility with existing systems
Exploitation of new environments
“System of Engagement”
IBM SoftLayer
IBM SCE+
Traditional IT Internet scale wkloads

SoftLayer provides world-wide services with a standardized modular
infrastructure; triple network architecture and powerful automation.
51
World-Wide Services
13 Data Centers
with 100,000 Servers and 22,000,000 Domains
in the US, Amsterdam and Singapore
19 Network Points of Presence
in 5 countries to facilitate response times
21,000 Customers
* Sold in US English, US $ Pricing
Tokyo
Hong Kong
Singapore
Seattle
San Jose
Los Angeles
Chicago
Denver
Dallas (6)
Houston (2)
New York City
Washington DC
Atlanta
Miami
Amsterdam
London
Frankfurt
Flexible, Automated Infrastructure
Data Center & Pods
• Standardized, modular hardware configurations
• Globally consistent service portfolio
Triple Network
• Public network for cloud services
• VPN for secure management
• Private network for communications and shared services
IMS (Automation Software)
• Bare metal provisioning
• Integrated BSS/OSS
• Comprehensive network management

Learning Points
 Cloud is being driven not only by cost,
but more importantly by:
52
– Time-to-market
– Elasticity
– Change business process
– Competitive imperatives
 Cloud is a significant shift in:
– Organizational lines
– Processes
– Workflows
– Workload types
– Required skill sets
 Cannot deliver true cloud services with
a traditional IT organization
– The workflow, process, responsibility,
reporting lines all different in cloud
– To provide elastic capacity, self-service E2E
automation
 Changing focus from on-premise
(traditional IT) to off-premise (cloud)
 IBM Cloud Storage products / directions
include:
– Traditional IT (on-prem or off-prem):
• Smart Cloud Storage Access, TPC,
Storwize, XIV
• OpenStack exploitation
– Object Storage
• Software defined object storage
– Design for Fail, Cloud Native IT:
• OpenStack + XIV/Storwize
• Softlayer

 “Building a 21st Century Cloud Storage Service – Industry Best Practices”
(external customer conference presentation):
For more reading and reference, full decks by John Sing:
53
– http://www.slideshare.net/johnsing1/building21stcenturycloudstorageservicejohnsingv4
 “State of the Cloud - Internet Scale Data Center Workloads – Comparison
to Traditional IT”: (external customer conference presentation):
– http://www.slideshare.net/johnsing1/s-ge01-toinfinityandbeyond2012bigdatainternetscaleupdatev2johnsing- “Disruptive Innovation in the Modern IT World”:
– http://www.slideshare.net/johnsing1/a-india-csii2012disruptiveinnovationinthemodernitworldv3plenarypresentation
 “Hadoop – it’s not just Internal Storage”:
– http://www.slideshare.net/johnsing1/hadoopitsnotjustinternalstoragev14

54
Gracias
Tesekkurler Turkish
Grazie
Hebrew
Russian
Thank You Japanese
Spanish
French
German
Italian
English
Brazilian Portuguese
Arabi
c
Traditional Chinese
Simplified
Chinese
Hindi
Tamil
Korean
Thai
German
Obrigado

55

56
Appendix: Disruptive Innovation

Cloud / mobile
market value
*bigger increases*
With all this opportunity……. Why is this Disruptive Change
flat-lining traditional consumer PC / desktop manufacturers?
57
 PC / laptop stalwarts
 Unsuccessful in shift
 To mobile
http://gigaom.com/2012/09/01/hp-dell-and-the-paradox-of-the-disrupted/
PC/laptop
market value
big decreases
noit azil ati paC t ekr a M

Observe: how fast mobile internet grows by 2014
 By 2014:
 Mobile will be
main way
 Of connecting to
Internet
58
Inter-
Disciplinary
http://www.digitalbuzzblog.com/2011-mobile-statistics-stats-facts-marketing-infographic

59
Disruptive Innovation
Definition:
 Create new
market and value
 Eventually
disrupts existing
 Displaces earlier
technology
Clayton Christensen
Harvard Business School
http://en.wikipedia.org/wiki/Disruptive_innovation

60
 Not “advanced
technologies”
 Inferior yet “good
enough”
 Novel combinations
 Starts low end
 Grows up-market
–“low end
disruption”
Clayton Christensen

61
 Learn lessons
Watch today’s
world
Illustrative examples only

 “Consumerization”
 Not just technology
 Delivery models
(cloud)
 Business models
 Ecosystems
62
Clayton Christensen

Mobile has affected all business models…
63
Mobile =
Geo-locational superfood
Real-time analytics
http://www.digitalbuzzblog.com/2011-mobile-statistics-stats-facts-marketing-infographic

Cloud-scale Data Centers required for:
Weatherbug
64
Data Supertransformagicability
TaxiWiz
HousingMaps
Source: http://mashable.com/2007/07/11/google-maps-mashups-2/

Web data,
video
70%
By 2016, how much mobile data? What kind?
65
 2012:
–Mobile-connected
devices > # people
 2016:
–10 billion mobile devices
–(world population: 7.3 B)
Smartphones
48%
http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html

66
 Big Data / Cloud on
disruptive path
 Traditional IT still
around but….
 Newer technologies
disrupt all platforms
Clayton Christensen
What will the effect be on
your IT organization?
Inter-
Disciplinary

Internet Scale Workload Characteristics - 1
67
 Embarrassingly parallel Internet workload
– Immense data sets, but relatively independent records being processed
• Example: billions of web pages, billions of log / cookie / click entries
– Web requests from different users essentially independent of each over
• Creating natural units of data partitioning and concurrency
• Lends itself well to cluster-level scheduling / load-balancing
– Independence = peak server performance not important
– What’s important is aggregate throughput of 100,000s of servers
i.e. Very low
inter-process
communication
 Workload Churn
– Well-defined, stable high level API’s (i.e. simple URLs)
– Software release cycles on the order of every couple of weeks
• Means Google’s entire core of search services rewritten in 2 years
– Great for rapid innovation
• Expect significant software re-writes to fix problems ongoing basis
– New products hyper-frequently emerge
• Often with workload-altering characteristics, example = YouTube

Internet Scale Workload Characteristics - 2
 Platform Homogeneity
 Fault-free operation via application middleware
 Immense scale:
68
– Single company owns, has technical capability, runs entire platform
end-to-end including an ecosystem
– Most Web applications more homogeneous than traditional IT
– With immense number of independent worldwide users
1% - 2% of all
Internet requests
fail*
Users can’t tell difference
between Internet down and
your system down
Hence 99% good enough
– Some type of failure every few hours, including software bugs
– All hidden from users by fault-tolerant middleware
– Means hardware, software doesn’t have to be perfect
– Workload can’t be held within 1 server, or within max size tightly-clustered
memory-shared SMP
– Requires clusters of 1000s, 10000s of servers with corresponding PBs
storage, network, power, cooling, software
– Scale of compute power also makes possible apps such as Google Maps,
Google Translate, Amazon Web Services EC2, Facebook, etc.
*The Data Center as a Computer: Introduction to Warehouse Scale Computing, p.81 Barroso, Holzle
http://www.morganclaypool.com/doi/pdf/10.2200/S00193ED1V01Y200905CAC006

Internet Scale data center power components…
69
Image courtesy of DLB Associates: D. Dyer, “Current trends/challenges in datacenter thermal management—a facilities perspective,”presentation at ITHERM, San Diego, CA, June 1, 2006.
“The Data Center as a Computer: Introduction to Warehouse Scale Computing”, figure 4-1, p.40 Barroso, Holzle

Breakdown of data center
energy overheads
70
Image courtesy of ASHRAE “The Data Center as a Computer: Introduction to Warehouse Scale Computing”, figure 5-2, p.49 Barroso, Holzle
Chiller alone is
33% of the cost
UPS alone is
18% of
construction
cost
Physical cooling,
UPS dominates the
electrical power cost

71
construction cost of Internet Scale Data Center is
Power / Cooling
Facebook’s North Carolina Data Center Goes Live
Facebook – Prinville, Oregon
Has spent $1B on it’s data centers
Open Compute Project
Facebook:
Lulea, Sweden - 290K ? Reducing power
profile reduces
construction cost

Total Building Power consumed
---------------------------------------------
IT power consumed
Wow. Given that fact…..
Whose data centers are most
power efficient?
72
 Reducing power profile = lowers
initial CAPEX SIGNIFICANTLY
 Therefore, fundamental Internet
Scale Data Center goal is:
 Decrease Power Usage
Effectiveness (PUE)
 PUE =
http://gigaom.com/cloud/whose-data-centers-are-more-efficient-facebooks-or-googles/

Google claims its data centers use
50% less energy than competitors
 Power Usage Effectiveness
73
– PUE=1.14 means power overhead is
only 14%
– Industry average is around 1.8
http://venturebeat.com/2012/03/26/google-data-centers-use-less-energy/
Industry average
PUE is about 1.8
http://www.datacenterknowledge.com/archives/2011/05/10/uptime-institute-the-average-pue-is-1-8/

C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing

Ähnlich wie C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing (20)

Mehr von John Sing

Mehr von John Sing (9)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

C cloud organizational_impacts_big_data_on-prem_vs_off-premise_john_sing

Hinweis der Redaktion