How do you optimize Apache Spark workloads in the cloud? How do you tune your resources for maximum performance and efficiency? Find out how the new Hortonworks Flex support subscriptions enables IT agility and success in the cloud. We will cover: * Options for running Data Science, Analytics and ETL workloads in the cloud * Hortonworks support offerings including new Flex Support Subscription * How to run Cloud workloads more efficiently with SmartSense * Case study on the impact of SmartSense https://hortonworks.com/webinar/powering-big-data-success-cloud/

1. 1 © Hortonworks Inc. 2011 –2017. All Rights Reserved1 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Powering Big Data Success
On-Prem and In the Cloud
Anna Yong, Director Product Marketing
Paul Codding, Director Product Management

2. 2 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Powering Big Data Success: On-Prem and In The Cloud
Anna Yong
Director Product
Marketing
Paul Codding
Director Product
Management

3. 3 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Hortonworks Connection Ensures Success of Your Big Data Journey

4. 4 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Big Data Success
hortonworks.com/solutions/big-data-success/

5. 5 © Hortonworks Inc. 2011 –2017. All Rights Reserved
New Hortonworks Flex Support Subscription
Universal, Usage-based
Support Subscription
Cloud & On-Prem
HDCloud
IaaS
On-Prem
Single, flexible, portable support
for transition to cloud
New support offering for Spark
Data Science, ETL, EDW-Analytics
in the Cloud
Performance optimization with
Hortonworks SmartSense

6. 6 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Hortonworks Data Cloud for AWS
à Pre-tuned for use with AWS
à Powered by HDP
à Focused on business agility
à Prescriptive, ephemeral use cases
• Data Science (Apache Spark & Zeppelin)
• Analytics (Apache HIVE)
• ETL (Apache HIVE & Spark)
• Business Intelligence (OLAP/Druid)

7. 7 © Hortonworks Inc. 2011 –2017. All Rights Reserved
EASE OF USE: Choose from a set of pre-
tuned and pre-configured templates.

8. 8 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Choose from a Set of Prescriptive, Ephemeral Workload Clusters
Data Science
Spark 1.6, 2.1
Business
intelligence
(OLAP) Druid TP
Analytics &
Reporting with
HIVE 2.2 & LLAP
ETL with Spark

9. 9 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Try it! Hortonworks Data Cloud for AWS Marketplace
How –to Video: https://hortonworks.com/video/hd-cloud-aws/
Try it Now: https://aws.amazon.com/marketplace/pp/B01LXOQBOU

10. 10 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Flex Support Subscription Components

11. 11 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Access to Expertise Plus Freedom, Agility and Flexibility
Expertise for Data Science, ETL and Analytics in HDCloud
Match support costs to usage patterns
Migrate infrastructure on your own terms
Support for Dev, QA & ephemeral production
Built-In Performance Optimization

12. 12 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Access to Expertise Plus Freedom, Agility and Flexibility
Expertise for Data Science, ETL and Analytics in HDCloud
Match support costs to usage patterns
Migrate infrastructure on your own terms
Support for Dev, QA & ephemeral production
Built-In Performance Optimization

13. 13 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Hortonworks Connected Data Platforms and Solutions
Data Services
Hortonworks Solutions
Enterprise Data
Warehouse Optimization
Cyber Security and
Threat Management
Internet of Things
and Streaming Analytics
Data Center
Hortonworks Data Suite
HDFHDP
Hortonworks
Connection
Cloud
Hortonworks Data Cloud
AWS HDInsight
Hortonworks Connection
Enablement Subscription
SmartSense™
Premier Operational Support
Educational Services
Professional Services
Community Connection

14. 14 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Hortonworks Connection Ensures Success of Your Big Data Journey

15. 15 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Accelerate Case Resolution
Prevent Issues
Understand Your Cluster

16. 16 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

17. 17 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

18. 18 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

19. 19 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

20. 20 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

21. 21 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

22. 22 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Accelerate Case Resolution
SmartSense provides Hadoop Operators with an Ambari Integrated tool to quickly capture
diagnostic information for specific services and hosts into a single “Bundle” that’s automatically
uploaded to Hortonworks Support.
Significantly reduces the back-and-forth nature of troubleshooting issues.
A M B A R I
O PS
H O R TO N W O R KS
S U P P O R T
S U P P O R T
C A S E
S m a r t S e n s e
S E R V E R
B U N D L E
GAT E WAY

23. 23 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Accelerate Case Resolution
Prevent Issues
Understand Your Cluster

24. 24 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Prevent Issues
SmartSense analyzes Bundles for configuration issues – recommendations are produced and
made available for each cluster in the Hortonworks Support Portal
Recommendations prevent operational issues, and improve performance and overall cluster
throughput.

25. 25 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Prevent Issues
SmartSense analyzes Bundles for configuration issues – recommendations are produced and
made available for each cluster in the Hortonworks Support Portal
Recommendations prevent operational issues, and improve performance and overall cluster
throughput.

26. 26 © Hortonworks Inc. 2011 –2017. All Rights Reserved
After Applying only 3 SmartSense Recommendations
They can now run 1200 concurrent jobs
...with only 350 waiting jobs at peak hours
Issue: YARN @ capacity, struggling to add
more use cases
Before SmartSense
à Could only run 500 jobs concurrently
à 1100 jobs would be pending waiting for
resources at peak hours
With SmartSense = 2X Throughput Improvement

27. 27 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Accelerate Case Resolution
Prevent Issues
Understand Your Cluster

28. 28 © Hortonworks Inc. 2011 –2017. All Rights Reserved
“Who’s creating all of these small files in HDFS!?”
“What are my top 10 most active users, and longest running
jobs?”
“How much should I charge users for their cluster
resource use?”
SmartSense Today – Understand Your Cluster

29. 29 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Understand Your Cluster
Chargeback Reporting

30. 30 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Understand Your Cluster
Chargeback Reporting
HDFS Dashboards

31. 31 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Today – Understand Your Cluster
Chargeback Reporting
HDFS Dashboards
YARN Dashboards

32. 32 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Impact of Hortonworks SmartSense
0
200
400
600
800
1000
1200
1400
Without
SmartSense
With
SmartSense
Concurrent Jobs
B U N D L E
2X Throughput
Improvement
Address 30% of
Issues
Configuration Issues
Avoid 10% of
Sev1 Issues
Production Down
Single-Bundle
Case Resolution
25% of the Time
SmartSense
Troubleshooting Bundle

33. 33 © Hortonworks Inc. 2011 –2016. All Rights Reserved
Bundle Security

34. 34 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Data Capture Architecture
S E R V E R
GAT E WAY
AMB AR I
AGE N T AGE N T
AGE N TAGE N TAGE N T
AGE N T
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E

35. 35 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Data Capture Architecture
S E R V E R
GAT E WAY
AMB AR I
AGE N T AGE N T
AGE N TAGE N TAGE N T
AGE N T
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
Agent to Server: TLS

36. 36 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Data Capture Architecture
S E R V E R
GAT E WAY
AMB AR I
AGE N T AGE N T
AGE N TAGE N TAGE N T
AGE N T
B U N D L E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
Agent to Server: TLS
Bundle: AES 256/RSA 1024

37. 37 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Data Capture Architecture
L AN D I N G ZO N E
S E R V E R
GAT E WAY
AMB AR I
AGE N T AGE N T
AGE N TAGE N TAGE N T
AGE N T
B U N D L E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
S m a r t S e n s e
A n a l y t i c s
Agent to Server: TLS
Bundle: AES 256/RSA 1024
Landing Zone: SOC2 Certified

38. 38 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Data Capture Architecture
L AN D I N G ZO N E
S E R V E R
GAT E WAY
AMB AR I
AGE N T AGE N T
AGE N TAGE N TAGE N T
AGE N T
B U N D L E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
S m a r t S e n s e
A n a l y t i c s
Agent to Server: TLS
Bundle: AES 256/RSA 1024
Server to Gateway: TLS
Landing Zone: SOC2 Certified

39. 39 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Data Capture Architecture
L AN D I N G ZO N E
S E R V E R
GAT E WAY
AMB AR I
AGE N T AGE N T
AGE N TAGE N TAGE N T
AGE N T
B U N D L E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
W O R KE R
N O D E
S m a r t S e n s e
A n a l y t i c s
Agent to Server: TLS
Bundle: AES 256/RSA 1024
Server to Gateway: TLS
Landing Zone: SOC2 Certified
Gateway to Landing Zone: HTTPS (TLS 1.2) or SFTP (AES)

40. 40 © Hortonworks Inc. 2011 –2016. All Rights Reserved
Supported Platforms

41. 41 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense OS Support
RHEL 6 RHEL 7 Debian 7 SLES 11 SLES 12 Ubuntu 12 Ubuntu 14
SmartSense 1.4

42. 42 © Hortonworks Inc. 2011 –2017. All Rights Reserved
SmartSense Stack Support
HDP 2.6 HDP 2.5 HDP 2.4 HDP 2.3
*2.3.4.7+
HDP 2.2
SmartSense 1.4
Ambari 2.5
Mandatory
Ambari 2.4
Built-In!
Ambari 2.2
Built-In!
Ambari 2.1
Plug-In
Ambari 2.0
Plug-In
SmartSense 1.4
HA & Kerberized Clusters Are Supported

43. 43 © Hortonworks Inc. 2011 –2016. All Rights Reserved
Getting Started

44. 44 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Getting Your SmartSense ID
Used to:
1. Link each bundle generated with your
Hortonworks Support Account
2. Attribute diagnostic information for each
cluster to the recommendations
generated by SmartSense
Every Hortonworks Support Customer can generate a SmartSense ID

45. 45 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Getting Your SmartSense ID
Used to:
1. Link each bundle generated with your
Hortonworks Support Account
2. Attribute diagnostic information for each
cluster to the recommendations
generated by SmartSense
Every Hortonworks Support Customer can generate a SmartSense ID

46. 46 © Hortonworks Inc. 2011 –2017. All Rights Reserved
Questions?
References
hortonworks.com/solutions/big-data-success/
hortonworks.com/services/support/flex/
hortonworks.com/services/support/smartsense/
hortonworks.com/products/cloud/aws/