SAP HANA SPS09 - Dynamic Tiering

1
©2014 SAP SE or an SAP affiliate company. All rights reserved.
SAP HANA SPS 09 - What’s New? HANA Dynamic Tiering
SAP HANA Product Management November 2014
(Delta from SPS 08 to SPS 09)

2
Public
Disclaimer
This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP.
SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP’s strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice.
This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.

3
Public
Agenda
Positioning
What is “SAP HANA Dynamic Tiering”, and what is its value to the customer?
Technical Details
Implementation choices
Use Cases
SAP BW and native HANA applications
Future Direction
Where are we headed?

Positioning
What is “SAP HANA Dynamic Tiering”, and what is its value to the customer?

5
Public
IDC predictions for 2014
Data explosion Data volumes will continue to explode to 6 billion petabytes
Social networking Social networking will become embedded in cloud platforms and most enterprise apps and processes
Cloud Cloud spending will surge by 25%, reaching over $100 billion. There will be a doubling of cloud data centers.
Internet of Things 30 billion devices, sensors in 2020 – driving $8.9 Trillion in revenue
Mobile
CRM Data
Planning
Opportunities
Transactions
Customer
Sales Order
Things
Instant Messages
Demand
Inventory
Big Data
Sales Order
Things
Mobile
Demand
Big Data
CRM Data
Customer
Planning
Transactions

6
Public
SAP End to End Data Management for Real Time Business
Business & Consumer Applications
Big Data
SAP DATA MANAGEMENT
STORE
TRANSACT
PREDICT
ANALYZE
Custom Development
ISVs & OEMs
ERP
Internet of Things
Workforce of the Future
Cloud
Industries

7
Public
e
SAP HANA platform
Processing Engine
Application Function Lib. & Data Models
Integration Services
SAP HANA PLATFORM
Real-time transactions + end-to-end analytics
Operational Analytics
Big Data Warehousing
Predictive, Spatial & Text Analytics
REAL-TIME ANALYTICS
Sense & Respond
Planning & Optimization
Consumer Engagement
REAL-TIME APPLICATIONS
SAP ESP
SAP ASE
Replication Server
SAP SQL Anywhere
SAP IQ
SAP Data Services
Extended Application Services
SAP Data Management Portfolio End-to End Data Management & App Platform for Real-Time Business
Database Services
SAP HANA dynamic tiering

8
Public
Time Value of Data
Time
Value
Last time accessed
Value of immediate data access declines
When you need it again
Archive Access Event
•Regulatory audit
•Business critical reference data
•Source data

9
Public
Multi-Temperature Storage Options with SAP HANA
Data Temperature
Storage Option
SAP BW on HANA
SAP Business Suite on HANA
SAP HANA Native
hot
SAP HANA In-Memory



cold
SAP HANA dynamic tiering
(1)

(2)
Data Aging (Next Gen ILM)

3

Near-line Storage (NLS)



frozen
Data Archiving (ADK)



 Generally available
 Combination not available
1 Early shipment available for SAP BW 7.4; General availability planned Q4/2014
2 General availability with limited scope planned Q4/2014
3 For selected business objects

10
Public
SAP HANA Dynamic Tiering Key aspects at a glance
Add-on Product to SAP HANA
Manage data of different temperatures
Hot data (always in memory) – classical HANA
Cold data (disk based data store)
Introducing a new type of table:
Extended table – disk-based columnar table
SPS 9 release focus
Operational integration
Common installer
Unified monitoring and administration
Integrated backup/recovery
Initial functional scope
Transparent query processing
Cross-store optimizer
Use extended table in calculation views
Applications manage data temperatures (no active support for aging)
SAP HANA Database
Data for daily reporting, other high-priority data
Other data required to operate the application
Hot
Warm

11
Public
Introducing SAP HANA Dynamic Tiering Requirements from our customers
Manage data cost effectively, yet with desired performance based on SLAs
Handle very large data sets – terabytes to petabytes
Update and query all data seamlessly via HANA tables
Application defines which data is “hot”, and which data is “warm”
Native Big Data solution to handle a large percentage of enterprise data needs without Hadoop
SAP HANA hot store (in-memory)
SAP HANA warm store (dynamic tiering)
Extended table (definition)
Extended table (data)
Fast data movement and optimized push down query processing
All data of extended table resides in warm store
SAP HANA Database System
Hot table (definition/data)

12
Public
Hot/Warm Data Management Questions about SAP HANA Dynamic Tiering
Size and cost constraints may prohibit all in-memory solution
Not all data has the same value
Warm data has lower latency requirements than hot data
Why is warm data management important for SAP HANA?
SAP HANA dynamic tiering utilizes disk backed, smart column store technology based on Intellectual Property from SAP Sybase
SAP HANA dynamic tiering excels at ad hoc queries on structured data from terabyte to petabyte scale
SAP HANA dynamic tiering is a deeply integrated, high performance solution in a single system
Why is SAP HANA dynamic tiering the best solution for warm data management?
Hadoop has unlimited capacity for raw data processing
Hadoop is best suited for batch processing of raw, unstructured data
Hadoop is an external data store with technical integration into HANA – with higher TCO in order to manage the additional system
What about Hadoop for warm data storage and processing?

13
Public
SAP HANA Dynamic Tiering Key aspects at a glance
Data in the database
Different data temperatures
Maximum access performance Hot data - always in memory
Reduced access performance: Warm data - not (always) in memory
All part of the database’s data image
Data moved out of the database
Different data qualities
Available for read access BW Near-line storage
Not accessible without IT process Traditional archive
Data is stored and managed outside of the application database
SAP HANA Database
Data for daily reporting, other high-priority data
Other data required to operate the application
Hot
Warm
NLS
Data that is (normally) not updated, infrequently accessed
Traditional Archive
Data that‘s kept for legal reasons or similar
Externalize

14
Public
Problems with temperatures There are too many options – across system boundaries
In DB
In memory
No restrictions, all features available
External to DB
Near-line Storage
Read access, no updates
In DB
On disk
hot
warm
cold
???
External to DB
Archive storage
No read access or updates
Performance and Price
Priority and Data Volume
HANA
Archive
HANA column and row store
Warm store of dynamic tiering / Non-Active Data Concept
BW Near-line Storage
Traditional Archive

15
Public
Problems with temperatures There are too many options – across system boundaries
In DB
In memory
External to DB
Near-line Storage
Read access, no updates
In DB
On disk
hot
warm
cold
???
External to DB
Archive storage
No read access or updates
Performance and Price
Priority and Data Volume
HANA column and row store
Warm store of dynamic tiering / Non-Active Data Concept
BW Near-line Storage
Traditional Archive
hot
warm
BW NLS
Archive

16
Public
SAP HANA Dynamic Tiering Map data priorities to data management
Hot Store
Classic HANA tables
Primary data image in memory
DB algorithms optimized for in-memory data
Persistence on disk to guarantee durability
Warm Store
Extended Tables
Primary data image on disk
Data processing using algorithms optimized for disk-based data
Main memory used for caching and processing.
SAP HANA Database
Primary image in memory
Durability
Cache / Processing
Primary Image on disk
Dynamic Tiering
Hot data
Warm data
All in one database
Hot Store
Warm Store
RAM

Technical Details
Implementation choices

18
Public
SAP HANA Dynamic Tiering – one database / one experience for HANA application developers and admins
SAP HANA Dynamic Tiering
Reduced TCO
Optimized for performance
Single database experience
Centralized operational control
Centralized monitoring / admin
High speed data ingest
Common installer and licensing model
Unified backup and restore
Integrated security
Optimized query processing
SAP HANA Dynamic Tiering

19
Public
SAP HANA Dynamic Tiering The overall system layout
SAP HANA with Dynamic Tiering consists of two types of hosts:
Regular worker hosts (running the classical HANA processes: indexserver, nameserver, daemon, xsserver,…)
–HANA hosts can be single-node or scale-out; appliance or TDI
“ES host” (running nameserver, daemon, and esserver)
–esserver is the database process of the warm store
One single SAP HANA database: one SID, one instance number
All client communication happens through index server / XS server
Hot Store
Fast data movement and optimized push down query processing
SAP HANA System with dynamic tiering service
Worker host(*)
Worker host
Worker host
Client Application
Connect
ES host
Column Table
Row Table
Extended Table
Warm Store
Common Storage System
(*) Standby hosts not shown

20
Public
Database Catalog
HANA Extended Tables
HANA Database
Warm Store
Data
HANA extended table schema is part of HANA database catalog
HANA extended table data resides in warm store
HANA extended table is a first class database object with full ACID compliance
Hot Store
Table Definition
Data
Table Definition
Classical HANA column/row table
Extended table (warm table)

21
Public
High Speed Data Ingest
Import from CSV files:
IMPORT FROM CSV FILE ‘bigfile.csv’ INTO t1
Bulk array insert:
INSERT INTO t1 (col1, col2, col3...) VALUES (val1, val2, val3...)
High-speed data movement between HANA tables and HANA extended tables:
INSERT INTO t_extended select c1 FROM t_hana
Concurrent inserts from multiple connections:
A HANA extended table may be a DELTA enabled table, which allows multiple concurrent writes
Warm Extended Table
IMPORT FROM CSV FILE ‘data.csv’ INTO t_extended
CSV
DATA
Hot HANA column Table
INSERT...SELECT
Materialization
Data movement between hot and warm store
HANA Database

22
Public
Optimized Query Processing
Parallel query processing
•Data is pulled from HANA hot store into HANA warm store query processing engine using multiple streams, and processed in parallel
Push/Pull query optimization and transformation
•Query operations ship to hot or warm store as appropriate for native performance
Extended tables may be used in HANA CALC views
•HANA Calc engine and HANA SQL engine share extended table query performance optimizations
Joining
Grouping
Ordering
T3
T4
T1
T2

23
Public
Example Query Plan
select
"account_num",
count(*) as account_count
from
VXM_FOODMART.CUSTOMER C
where
"lname" >= 'Ga' and "lname" < 'Gb'
and exists
(
select *
from
VXM_IQSTORE.PRODUCT P
where
"product_id" = "customer_id"
)
group by
"account_num"
order by
"account_num";
Customer is a native HANA table in HANA memory
Product is a HANA extended table in the warm store

24
Public
HANA Monitoring and Administration
HANA Cockpit:
New, web based monitoring and administration console for HANA Extended Storage
HANA Studio will be used for design and modeling of HANA extended tables
HANA Cockpit displays status, CPU/memory/storage resource utilization, table usage statistics
Provides access to and search of server logs and custom traces
Shows alerts triggered by extended storage
Enables administration of extended storage: add and drop storage, or increase size of file

25
Public
Unified Backup and Restore
HANA backup manages backup of both hot and warm store
Point in Time Recovery (PITR) is supported
Extended Storage
HANA
Data backups (manual or scheduled)
Log backups (automatic, or none)
Data backup
Log backup
System crash
Restore
Time
t1
t2
t3
Data backups with log backups allow restore to Point in Time or most recent state: t1- > t3
Data backups alone allow restore to specific backup only: t1 or t2
Log area
Backup History

26
Public
High Availability and Disaster Recovery
High availability
Compute node failure will result in failover to standby node (manual for warm store nodes)
Storage failure will depend on inherent storage vendor disk mirroring and fault tolerance capabilities
Hot and warm store should use the same storage to facilitate auto-failover in the future
Disaster recovery
HANA without Dynamic Tiering supports continuous replication to maintain a disaster recovery site
HANA with Dynamic Tiering will maintain a disaster recovery site through backup and restore capabilities only
–Disaster recovery through system replication is planned for a future release
–Disaster recovery through storage replication may be added independently from software releases
Classical HANA services
Compute node
Hot Store
Warm Store Service
Compute node
Standby node
Manual Failover
Standby node
Warm Store
Auto- Failover
mirror
mirror

27
Public
Support in SAP HANA multiple database containers (MDC)
MDC: One SAP HANA system can have multiple tenant databases
Each tenant database can be associated with zero or one extended stores
Each extended store is dedicated to exactly one tenant database
SAP HANA system with MDC and dynamic tiering
Compute node
System Database
Compute node
Compute node
Tenant Database <B>
Extended Store
Tenant Database <A>
Tenant Database <C>
Extended Store
Classical HANA (single-node or scale-out)
ES Host <B>
ES Host <C>

Use Cases
SAP BW and native HANA applications

29
Public
SAP NetWeaver BW powered by SAP HANA Data Classification by Object Type
Frequent reporting and/or HANA-native operations
BW – Operational Data
Data Categories in a BW System
Staging Layer
Analytic Mart
Business Transformation
EDW Propagation
EDW Transformation
Corporate
Memory
Archive/NLS
“Old”, “out-of-use” data – Archive, read-only, different SLAs
Limited reporting, limited HANA-native operations
Archived

30
Public
SAP HANA database
Database Catalog
Extended Tables in HANA BW Use Case: Staging and Corporate Memory
Object Classification in BW
Data Sources and write-optimized DSOs can have the property “Extended Table”
Generated Tables are of type “Extended”
All BW standard operations supported – no changes
Only minor temporary RAM required in HANA
InfoCubes and Regular or Advanced DSOs
Generate standard column table
Hot Store
Warm store
BW System
Corporate Memory
Write-optimized DSO
Staging Area
Data Source
Table Schema
Data
PSA Table
Table Schema
Data
Active Table
Data Mart
InfoCube
Table Schema
Data
Fact Table

31
Public
SAP HANA Dynamic Tiering for Big Data
Cutting edge, in-memory platform
Transact/analyze in real-time
Native predictive, text, and spatial algorithms
Petascale extension to HANA with disk backed, columnar database technology
Expand HANA capacity with warm/cool structured data in HANA warm store
Tight integration between HANA hot store and HANA warm store for optimal performance
SAP HANA with Dynamic Tiering provides native Big Data solution
Hot data
SAP HANA
Petascale, warm structured data
HANA extended tables

32
Public
SAP HANA with Dynamic Tiering Native Big Data solution for a multitude of use cases
SAP HANA Dynamic Tiering for Big Data Use Cases across Industries
Airline route profitability analysis: SAP HANA analyzes revenue, variable operating costs (fuel, landing fees...), and fixed operating costs in real time to make decisions on network, pricing, and marketing to determine where to fly, when, and how often. All data must be analyzed in real time.
Financial services: Stock tick data streamed into SAP HANA for immediate price fluctuation analysis and trading actions, with historical stock price data stored in HANA extended tables for trend analysis and portfolio management.
Telecommunications: Network service data in HANA extended tables analyzed and correlated with customer loyalty data in SAP HANA, to anticipate customer churn and initiate customer retention response activities.
Public utilities: enterprise data stored in SAP HANA and large amounts of smart meter data stored in HANA extended tables, to identify operational problems, and establish incentive pricing for more efficient energy use.

Future Direction
Where are we headed?

34
Public
SAP HANA Dynamic Tiering roadmap
SAP HANA dynamic tiering available to be used by any HANA application (if the application supports the feature)
Common installer
Unified administration and monitoring using HANA Cockpit
Extended Storage (ES) engine is part of HANA topology
Single authentication model
Single licensing model
Combined error log / trace handling
Integrated File-based backup/recovery, including point-in time recovery
HANA ES host scale-out and auto-failover (HA)
Disaster Recovery (SAP HANA system replication)
Further integration with respect to backup/recovery
Hybrid extended tables with rule based automatic data movement / aging
Optimization of communication between hot and warm store
Further unification of DDL and DML for HANA extended tables
Further optimizer enhancements
Further extension of unique HANA capabilities to warm store
FUTURE
PLANNED

35
Public
Hybrid extended tables
Automatic, rules-based, asynchronous data movement between hot and warm stores
Hot partitions in HANA memory; remaining partitions in warm store
Single HANA table that spans hot and warm stores
Hot data in HANA tier
Warm data In warm tier
2012
2012
Hybrid
Extended
Table
aging
regulatory audit

36
Public
How to find SAP HANA documentation on this topic?
•In addition to this learning material, you can find SAP HANA platform documentation on SAP Help Portal knowledge center at http://help.sap.com/hana_platform.
•The knowledge centers are structured according to the product lifecycle: installation, security, administration, development:
SAP HANA Options
SAP HANA Advanced Data Processing
SAP HANA Dynamic Tiering
SAP HANA Enterprise Information Management
SAP HANA Predictive
SAP HANA Real-Time Replication
SAP HANA Smart Data Streaming
SAP HANA Spatial
•Documentation sets for SAP HANA options can be found at http://help.sap.com/hana_options:
SAP HANA Platform SPS
What’s New – Release Notes
Installation
Administration
Development
References
•

Thank you
Contact information
Richard Bremer, Courtney Claussen, Balaji Krishna, and Robert Waywell
SAP HANA Product Management
AskSAPHANA@sap.com

38
Public
© 2014 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward- looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

SAP HANA SPS09 - Dynamic Tiering

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (13)

Ähnlich wie SAP HANA SPS09 - Dynamic Tiering

Ähnlich wie SAP HANA SPS09 - Dynamic Tiering (20)

Mehr von SAP Technology

Mehr von SAP Technology (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

SAP HANA SPS09 - Dynamic Tiering