The rise of big data and the Apache Hadoop platform allows for the capture and processing of data at an unprecedented scale and velocity. Watch this slide deck to get a comprehensive overview of the Apache Hadoop platform architecture and learn how to leverage the strengths of both the Apache Hadoop and SAP HANA platforms.

1. ©2014 Method360, Inc. All rightsreserved.
1
Leveraging SAP HANA with Apache Hadoop
Presented By
Bing Chen | Director

2. ©2014 Method360, Inc. All rightsreserved.
2
Method360
About Us
Leading Information Technology Consulting Services Company
•Headquartered in San Francisco, CALocal offices in San Diego, Chicago, Dallas, Pittsburgh, and Calgary
•Celebrating Our 15th Year of Operations
•100+ Employees Throughout the US
•Focus on Data, Analytics and Quality Assurance
•Strong Emphasis on Consultant Quality and Client Relationship
•Onsite, Offsite and NearshoreCapabilities
•Certified SAP Services Partner
COMPANY OVERVIEW

3. ©2014 Method360, Inc. All rightsreserved.
3
Method360
SAP HANA WITH APACHE HADOOP
Introduction
Bing Chen
Director
Advanced Analytics Practice

4. ©2014 Method360, Inc. All rightsreserved.
4
Method360
SAP HANA WITH APACHE HADOOP
This session explores Apache Hadoop and how to leverage it with
SAP HANA and SAP analytics to drive new solutions and solve
business problems
•Overview of the Hadoop platform architecture
•Differences between SAP HANA and Hadoop
•Key analytics scenarios and use cases leveraging Hadoop, SAP HANA, and SAP analytics
•Live demo of SAP HANA and Hadoop scenarios
In This Session

5. ©2014 Method360, Inc. All rightsreserved.
5
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

6. ©2014 Method360, Inc. All rightsreserved.
6
Method360
SAP HANA WITH APACHE HADOOP
Big Data
•“Big Data” has many interpretations, but commonly understood as a combination of the following three “Vs”: VolumeVelocityVariety
•Requires new generation of techniques and technologies to capture, store, manage, and analyze the information
•Traditional RDBMS have challenges with Big Data
•Data Veracity also must be addressed –Data source and accuracy

7. ©2014 Method360, Inc. All rightsreserved.
7
Method360
SAP HANA WITH APACHE HADOOP
What is Hadoop?
•Apache Hadoop is an open-source software framework for storage and large scale processing of data sets on clusters of commodity hardware
•Framework composed of the following core modules: Hadoop Common
oCore libraries and utilitiesHadoop Distributed File System (HDFS)
oDistributed file system that stores data on commodity machinesHadoop YARN
oResource management and schedulingHadoop MapReduce
oProgramming model for large scale data processing

8. ©2014 Method360, Inc. All rightsreserved.
8
Method360
SAP HANA WITH APACHE HADOOP
Cluster of Commodity Servers
•Hadoop runs on hundreds or thousands of low-cost commodity serversMassively parallel batch processing broken down to run subsets of data processing on “DataNodes” “NameNode” manages data, replication, job processing on “DataNodes” Add “DataNodes” to scale to Petabytes
NameNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNodeDataNode

9. ©2014 Method360, Inc. All rightsreserved.
9
Method360
SAP HANA WITH APACHE HADOOP
MapReduceProgramming Model
•MapReduceis used to execute data processing and analytics jobs in parallel
•Two parts to these jobs: Map phase –Divides the job into smaller individual units of work with partial resultsReduce phase –Combines partial results from MapReducephase into final result
•Hadoop manages job distribution, restarts

10. ©2014 Method360, Inc. All rightsreserved.
10
Method360
SAP HANA WITH APACHE HADOOP
Other Hadoop-Related Components
•Other Apache projects also often considered as part of Hadoop platform: HBase
oAllows rapid read/write of large documents in HadoopHive
oSQL-like read-only queries (HiveQL) Pig
oGenerate procedural MapReduceprograms (Pig Latin) Mahout
oStatistical and analytics library to run on HadoopSqoop
oTransfer bulk data between Hadoop and structured data stores

11. ©2014 Method360, Inc. All rightsreserved.
11
Method360
SAP HANA WITH APACHE HADOOP
SAP and Hadoop Partnerships
•Commercial Open Source makes Hadoop more easily implemented and supported in the Enterprise
•SAP has reseller agreements and partnerships with several vendors
•SAP has been certified on several of these platforms

12. ©2014 Method360, Inc. All rightsreserved.
12
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

13. ©2014 Method360, Inc. All rightsreserved.
13
Method360
SAP HANA WITH APACHE HADOOP
Platform Comparison
•What are the key differences between RDBMS, In-Memory (SAP HANA), and Hadoop? RDBMSwill continue to have a place in existing systems like OLTP and conformed EDWSAP HANA is best suited for high-speed data and analytics scenarios, but cost can be high with very large data volumesHadoopis well suited for very large, unstructured data scenarios, but data processing and analysis can be very slow

14. ©2014 Method360, Inc. All rightsreserved.
14
Method360
SAP HANA WITH APACHE HADOOP
Hadoop vs. SAP HANA
Hadoop
SAP HANA
Data Architecture
Unstructured data and files on disk
Structured data in memory
Data Structures
No predefined schema
Predefined schemas and models
Performance
Slow data access, seconds to hours
Very fast, milliseconds to seconds
Scalability
Scale-out to thousands of commodity nodes
Scale-up/Scale-out to many servers
Data Consistency
BASE (Basic Availability, Soft state, Eventual consistency)
ACID (Atomicity, Consistency, Isolation, Durability)
Licensing Cost
Free open source or commercial open source
Many options from cloud to enterprise
Storage Cost and Type Flexibility
Speed and Data Conformity

15. ©2014 Method360, Inc. All rightsreserved.
15
Method360
SAP HANA WITH APACHE HADOOP
When to Use Hadoop vs. HANA?
•Hadoop has the lowest storage cost and highest data type flexibility, but also the slowest processing speed
•SAP HANA has highest processing speed and data conformity, but is more limited by cost and data type
•Key is to leverage strengths of both platforms
oHadoop + SAP HANA: Infinite Storage and Instant Insight!

16. ©2014 Method360, Inc. All rightsreserved.
16
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

17. ©2014 Method360, Inc. All rightsreserved.
17
Method360
SAP HANA WITH APACHE HADOOP
Key Scenarios
•Hadoop as Flexible Data Store
•Hadoop as Processing Engine
•Hadoop and SAP HANA for Analytics

18. ©2014 Method360, Inc. All rightsreserved.
18
Method360
SAP HANA WITH APACHE HADOOP
Hadoop as Flexible Data Store
•Use Hadoop to capture all types of data from multiple sourcesSAP and non-SAP, internal and external sourcesFull fidelity, lowest level granularity capture, and storage of data of any type allows preservation of data for future useStore and retrieve very large data sets and objectsAggregate and consolidate OLTP data in Hadoop to create OLAP fact tables for SAP HANAFeed SAP HANA, SAP BusinessObjects, Predictive Analysis via Hive, or Data Services ETLInteractive Big Data Exploration

19. ©2014 Method360, Inc. All rightsreserved.
19
Method360
SAP HANA WITH APACHE HADOOP
Flexible Data Store
HadoopSAP HANASAP BI SAP Data ServicesDiscovery and Query on Hadoop Data ETLETLFederated Smart Data AccessTransferDataReporting on combined HANA and Hadoop DataOther AppsReportingVisualization Real-Time Data Acquisition Streaming DataMachine and server logsSocial MediaOLTP Transaction DataDocuments, EmailMedia and BLOBsReference DataOLTP Transaction DataOther DatabasesFlat FilesBatch Data Feeds

20. ©2014 Method360, Inc. All rightsreserved.
20
Method360
SAP HANA WITH APACHE HADOOP
Flexible Data Store Examples
Data
Description
Data Stream Capture
Real-time capture of high-volume data streams, such as machine generated log and sensor data, real-time Web logs
Document and Multimedia Storage
Very high-volume storage of business documents (Healthcare, Insurance). Rapid high-volume storage and retrieval of media and BLOBs for social and Web applications like Facebook using HBase.
Social Media and Email
Real-time capture of social and email text data for sentiment analytics, email archiving
OLTP Transaction Data
Capture of high-volume OLTP transactions such as call center, inventory, and any other process transactions. Aggregate transactions and build OLAP fact tables for SAP HANA. ETL via SAP Data Services to SAP HANA.
Reference Data
Copy of existing large reference data sets such as GIS, survey, industry-specific data sets can be combined with other data for analytics
Data Archive
Archive of system logs, audit data, and other data that otherwise would go to long-term, off-site storage

21. ©2014 Method360, Inc. All rightsreserved.
21
Method360
SAP HANA WITH APACHE HADOOP
Hadoop as a Processing Engine
•Use Hadoop as a data processing engine for ETL rationalization to feed SAP HANAMapReduceprograms execute process logicPig for data analysisMahout for data mining and machine learningReplicate master data to Hadoop for data processingFeed results to SAP HANA with Data Services and merge with conformed data model

22. ©2014 Method360, Inc. All rightsreserved.
22
Method360
SAP HANA WITH APACHE HADOOP
Processing Engine
HadoopSAP HANA SAP Data Services ETLMapReducePigFederated Smart Data AccessParallel Load High Value Data Real-Time Data Acquisition Streaming DataMachine and server logsSocial MediaOLTP Transaction DataDocuments, EmailMedia and BLOBsReference DataOLTP Transaction DataOther DatabasesFlat FilesBatch Data Feeds

23. ©2014 Method360, Inc. All rightsreserved.
23
Method360
SAP HANA WITH APACHE HADOOP
Processing Engine Examples
Data
Description
Data Cleansing and Enrichment
Fix data issues in Hadoop, enhance with additional information
ETL Rationalization
Low-latency ingestion of data from operational systems
Tiered storage –High-value data loaded and transformed in HANA in parallel, off-load preprocessing to Hadoop
Data Mining and Predictive Analysis
Correlation, clustering, regression analysis. Predict machine failure, correlate customer behavior across systems.
Identify differences
Differences in large, but different sets of data such as DNA analysis
Risk Analysis
Fraud detection, identify risk patterns

24. ©2014 Method360, Inc. All rightsreserved.
24
Method360
SAP HANA WITH APACHE HADOOP
Hadoop and SAP HANA for Analysis
•How does Hadoop fit into the Data Analytics Process with SAP HANA and BI? Hadoop can store such high volumes of data that it often can’t be replicated into SAP HANA in a cost effective or timely manner
oSome of the analysis must be done in Hadoop, as well as SAP HANA
oQueries executed in Hadoop take much longer to run than SAP HANAAnalysis will likely require combining data from Hadoop, SAP HANA, and other data sources

25. ©2014 Method360, Inc. All rightsreserved.
25
Method360
SAP HANA WITH APACHE HADOOP
Two Approaches
•Differences between Hadoop and SAP HANA necessitate two fundamental approaches to combined analytics: Two-Phase Analytics
oAddresses long running Hadoop query times
oRun analysis continually on Hadoop, then periodic updates to SAP HANA for fast interactive query responseFederated Queries
oSplit analysis into parts and run asynchronously on Hadoop, SAP HANA, other systems
oFederate results in SAP HANA or BI

26. ©2014 Method360, Inc. All rightsreserved.
26
Method360
SAP HANA WITH APACHE HADOOP
Two-Phase Analytics
•Hadoop runs data mining, statistical analysis, OLAP fact table generation – “Slow” analytics
•SAP Data Services ETL process pushes results to SAP HANA for “Fast” analytics
CIO Guide: How to Use Hadoop with your SAP Software Landscape (SAP AG, February 2013).

27. ©2014 Method360, Inc. All rightsreserved.
27
Method360
SAP HANA WITH APACHE HADOOP
Federated Queries
•Split analysis into multiple queries, consolidate results
CIO Guide: How to Use Hadoop with your SAP Software Landscape (SAP AG, February 2013).

28. ©2014 Method360, Inc. All rightsreserved.
28
Method360
SAP HANA WITH APACHE HADOOP
Federation Scenarios
•Client-side federationBI tool queries separately and combines resultsOnly for smaller data and result sets
•Query federationServer-side execution of multiple queries and results combinedBetter for larger data sets
•Data federationHadoop data virtualized as a table by another database likeSAP HANA

29. ©2014 Method360, Inc. All rightsreserved.
29
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

30. ©2014 Method360, Inc. All rightsreserved.
30
Method360
SAP HANA WITH APACHE HADOOP
SAP HANA and Hadoop Integration
•Value derived from combining data from Hadoop with structured conformed data model in SAP HANA
•SAP HANA can integrate with HadoopSmart Data Access
oVirtual table created in SAP HANA points to remote Hive source, queries pushed down to HiveSAP Data Services
oConnect via Hive, HDFS
oPush MapReducejobs to Hadoop with Pig scripts

31. ©2014 Method360, Inc. All rightsreserved.
31
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

32. ©2014 Method360, Inc. All rightsreserved.
32
Method360
SAP HANA WITH APACHE HADOOP
Reference Architecture
HadoopSAP HANAHiveHBaseMahoutSqoopMapReduceHDFSSAP BI SAP Data Services Real-Time Data AcquisitionSAP SLTSAP Sybase Replication ServerSAP Sybase ESP…. BOBJLumiraPredictive AnalysisDashboardsMobile Analytics on Hadoop Data ETLETLFederated Smart Data AccessTransferDataOLAPPALBFLApp Server (XS) Analytics on combined HANA and Hadoop DataOther AppsBIMobile... HadoopOLTPDevStreaming DataMachine and server logsSocial MediaOLTP Transaction DataDocuments, EmailMedia and BLOBsReference DataOLTP Transaction DataOther DatabasesFlat FilesBatch Data Feeds

33. ©2014 Method360, Inc. All rightsreserved.
33
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

34. ©2014 Method360, Inc. All rightsreserved.
34
Method360
SAP HANA WITH APACHE HADOOP
Use Case —Preventative Maintenance
•Preventative maintenance of hardware to improve customer satisfaction
•Early failure identification and prediction
•Combine data from multiple sourcesMachine monitoring logsCall center dataCustomer surveysCRM dataManufacturing data

35. ©2014 Method360, Inc. All rightsreserved.
35
Method360
SAP HANA WITH APACHE HADOOP
Implementation — Preventative Maintenance
1.Use Hadoop to analyze call center and survey data
2.Combine results in SAP HANA with machine log data and run predictive models
3.Combine results with CRM, manufacturing data for reports
HadoopSAP HANAHiveSAP Predictive AnalysisETLOLAPPALBOBJ SAP Data Services ETLETLETLHDFSETLCall center logsCustomer SurveysMachine monitoring logsCRM dataManufacturing data

36. ©2014 Method360, Inc. All rightsreserved.
36
Method360
SAP HANA WITH APACHE HADOOP
Use Case —Data Warehouse Migration
•Migrate existing EDW to Hadoop + SAP HANA
•Legacy EDW not well suited to streaming, machine, and unstructured data
•Increase capacity, speed, and flexibility
•Migrate in multiple phases to keep continuous availability to EDW

37. ©2014 Method360, Inc. All rightsreserved.
37
Method360
SAP HANA WITH APACHE HADOOP
Implementation —
Data Warehouse Migration
1.Replicate data in Hadoop
2.ETL rationalization and data aggregation in Hadoop
3.Push aggregated data to Hadoop in parallel with existing EDW
4.Implement SAP BI on HANA and Hadoop
HadoopSAP HANAETLSAP BI SAP Data Services ETLETLETLLegacy EDWETLStreaming DataMachine and server logsOLTP Transaction DataReference Data

38. ©2014 Method360, Inc. All rightsreserved.
38
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

39. ©2014 Method360, Inc. All rightsreserved.
39
Method360
SAP HANA WITH APACHE HADOOP
Demo —Hadoop and SAP HANA

40. ©2014 Method360, Inc. All rightsreserved.
40
Method360
SAP HANA WITH APACHE HADOOP
What We’ll Cover
•What is Hadoop?
•Hadoop vs. SAP HANA
•Key scenarios leveraging SAP HANA and Hadoop
•SAP HANA and Hadoop integration
•Reference architecture
•Use case examples
•Demo of SAP HANA and Hadoop scenarios
•Wrap-up

41. ©2014 Method360, Inc. All rightsreserved.
41
Method360
SAP HANA WITH APACHE HADOOP
Where to Find More Information
•http://hadoop.apache.org/ Apache Hadoop site and resources
•www.saphana.com/docs/DOC-3777CIO Guide: How to Use Hadoop with your SAP Software Landscape (SAP AG, February 2013)
•www.sapbigdata.com/platform/hadoop/ Info and resources for running SAP with Hadoop
•http://developer.yahoo.com/hadoop/tutorial/ Hadoop tutorial

42. ©2014 Method360, Inc. All rightsreserved.
42
Method360
SAP HANA WITH APACHE HADOOP
7 Key Points to Take Home
•Three “Vs”of Big Data require new generation of technology and techniques
•Hadoop excels at very high-scale, low-cost/TB and data type flexibility
•SAP HANA excels at speed and structure
•Leverage strengths of both platforms in data store, data processing, and analytics scenarios
•Carefully evaluate your requirements and use cases against these scenarios
•Consider commercial open source and SAP partners for Hadoop distributions
•Leverage online Hadoop resources for training and POCs

43. ©2014 Method360, Inc. All rightsreserved.
43
Method360
SAP HANA WITH APACHE HADOOP
Your Turn!

44. ©2014 Method360, Inc. All rightsreserved.
44
Method360
SAP HANA WITH APACHE HADOOP
Question
Do you see Hadoop possibly taking over all of SAP HANA type advantages in the long run?

45. ©2014 Method360, Inc. All rightsreserved.
45
Method360
Contact Info
SVP, SALES
GREG CLARKE
714.380.2753
gclarke@method360.com
SALES
PRACTICES
BUSINESS INTELLIGENCE
BLAKE DUGGAN
415.548.0762
bduggan@method360.com
SAP HANA
BING CHEN
415.424.4533
bchen@method360.com
ENTERPRISE INFORMATION MANAGEMENT
JIM HANCK
415.694.8229
jhanck@method360.com
QUALITY ASSURANCE
SUNNY HIRAN
415.535.5453
shiran@method360.com
MANAGED SERVICES, PMO AND DELIVERY
ERIK AXTON
925.389.6708
eaxton@method360.com
FINANCIAL MANAGEMENT
GREG STIMPSON
925.640.8785
gstimpson@method360.com
SAP HANA WITH APACHE HADOOP

46. ©2014 Method360, Inc. All rightsreserved.
46
Method360
Thank You
END-TO-END SERVICES. HIGH VALUE SOLUTIONS.
SAP HANA WITH APACHE HADOOP