Cloudera Navigator Optimizer analyzes existing SQL workloads to provide instant insights into your workloads and turns that into an intelligent optimization strategy so you can unlock peak performance and efficiency with Hadoop.

1. 1© Cloudera, Inc. All rights reserved.
Unlock Hadoop Success with
Cloudera Navigator Optimizer
Anupam Singh | Head of Data Management | Cloudera

2. 2© Cloudera, Inc. All rights reserved.
Agenda
• Cloudera Enterprise Overview
• The Cost of Inefficient Workloads
• Cloudera Navigator Optimizer
• Live Demo
• Roadmap

3. 3© Cloudera, Inc. All rights reserved.
Cloudera Enterprise
Making Hadoop Fast, Easy, and Secure
A new kind of data
platform:
• One place for unlimited data
• Unified, multi-framework data
access
Cloudera makes it:
• Fast for business
• Easy to manage
• Secure without compromise
OPERATIONS
DATA
MANAGEMENT
STRUCTURED UNSTRUCTURED
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT SECURITY
FILESYSTEM RELATIONAL NoSQL
STORE
INTEGRATE
BATCH STREAM SQL SEARCH SDK

4. 4© Cloudera, Inc. All rights reserved.
Fast for Business
More Insights for More Users, at Lower TCO
Powerful Data Processing
The most Spark experience
MPP Analytic SQL, Built for BI
Lowest latency, best concurrency
Only Native Search
Complete accessibility
Updateable Analytic Storage
Simple real-time analytics and updates
Active Data Optimization
Rationalizes data and workloads for
peak performance
OPERATIONS
Cloudera Manager
Cloudera Director
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee
Optimizer
STRUCTURED
Sqoop
UNSTRUCTURED
Kafka, Flume
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
STORE
INTEGRATE
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Kite

5. 5© Cloudera, Inc. All rights reserved.
Active Data Optimization
Rationalize for Peak Performance on Hadoop with Navigator Optimizer
Cloudera Navigator Optimizer: Instantly
unlock your best Hadoop strategy
• Instant insights to understand existing workloads
• Intelligent guidance to optimize the right workloads
for Hadoop
• Recommendations to reduce Hadoop development
effort
Designed for Hadoop
• Efficiently support key Hadoop use cases
• Take advantage of compatibilities for Hive and Impala
• Prioritize based on impact
DATA
MANAGEMENT
Cloudera Navigator
Encrypt and KeyTrustee
Optimizer
STRUCTURED
Sqoop
UNSTRUCTURED
Kafka, Flume
PROCESS, ANALYZE, SERVE
UNIFIED SERVICES
RESOURCE MANAGEMENT
YARN
SECURITY
Sentry, RecordService
FILESYSTEM
HDFS
RELATIONAL
Kudu
NoSQL
HBase
STORE
INTEGRATE
BATCH
Spark, Hive, Pig
MapReduce
STREAM
Spark
SQL
Impala
SEARCH
Solr
SDK
Kite

6. 6© Cloudera, Inc. All rights reserved.
HOLD FOR POLLING QUESTION
Which problems are you currently experiencing with your data warehouse?
• Lack of visibility into SQL workloads
• ETL processing is pushing DW system to the limit
• Reporting takes a long time to complete
• Want to offload but not clear what to start with
• Too many queries are taking too long to run

7. 7© Cloudera, Inc. All rights reserved.
Inefficient data workloads cost money
Frequent ETL breakdowns
($$)
Long reporting wait times
($$$)
Ad hoc access pressure on EDW
($$$$)
Extreme query complexity
($)

8. 8© Cloudera, Inc. All rights reserved.
A day in the life of Data Management
12 am 12 pm 12 am
800K
600K
400K
200K
6 am 6 pm

9. 9© Cloudera, Inc. All rights reserved.
… driven with different Business Needs
12 am 12 pm 12 am
800K
600K
400K
200K
6 am 6 pm
ETL data prep BI reports Data Science Handwritten
1. Frequent ETL breakdowns
1. Long reporting wait times
1. Ad hoc pressure on EDW
4. Extreme query complexity

10. 10© Cloudera, Inc. All rights reserved.
We build workload optimization strategies
ETL data prep BI reports Data Science Handwritten
1. ETL
Consolidate update flows into
simplified update flow
2. Reporting
Materialize logically similar
steps for multi-step reports
3. Ad hoc
De-duplicate and recommend
top queries for Hadoop
4. Handwritten
Refactor complex legacy
queries into manageable pieces
800K
600K
400K
200K
12 am 12 pm 12 am6 am 6 pm

11. 11© Cloudera, Inc. All rights reserved.
… that save customers time and money
ETL data prep BI reports Data Science Handwritten
-206 hrs
-166 hrs
- 300 hrs
-120 hrs
1. ETL Consolidation
1. BI Optimization
3. Query De-Duplication
4. Query Redesign
800K
600K
400K
200K
12 am 12 pm 12 am6 am 6 pm

12. 12© Cloudera, Inc. All rights reserved.
Cloudera Navigator Optimizer
Unlock Your Best Hadoop Strategy, Instantly

13. 13© Cloudera, Inc. All rights reserved.
Total Customer Queries Uploaded in 6mos
>1.5M

14. 14© Cloudera, Inc. All rights reserved.
Total Customer Savings Generated to Date
>$10M

15. 15© Cloudera, Inc. All rights reserved.
What does it take to optimize workloads?

16. 16© Cloudera, Inc. All rights reserved.
Deep compiler tech to optimize thousands of queries
Thousands of queries in minutes
Distributed Compilation
Identify relationships across queries
Flightpath
Query Similarity Index
Flows
Query Rewrite Engine
HAQR

17. 17© Cloudera, Inc. All rights reserved.
HOLD FOR POLLING QUESTION
What top insights and features would be useful for getting started with Hadoop?
• Identify tables/columns and top data usage
• Query duplication (top queries)
• Query complexity (joins, subqueries, etc)
• Recommendations for what and how to offload
• Hadoop compatibility and query rewrite tools
• Identify data flow for ETL and/or reporting

18. 18© Cloudera, Inc. All rights reserved.
The DCC Rule
D C C

19. 19© Cloudera, Inc. All rights reserved.
The DCC Rule
D C C
Complexity
Maximize your optimization
opportunities by exposing
complex access patterns that
make the best use of Hadoop’s
architecture
Compatibility
Reduce development time by
leveraging existing query
compatibilities with Hadoop
tools and get guidance for
query rewrites
Duplication
Improve performance by easily
detecting workload duplication
and recommending top queries
to optimize

20. 20© Cloudera, Inc. All rights reserved.
Demo

21. 21© Cloudera, Inc. All rights reserved.
Roadmap
• Platform Integration
• Enterprise: Security, governance, administration
• SQL-on-Hadoop frameworks
• Workload Optimization Completion across Use Cases
• BI reporting, self-service BI, ETL
• Impala, Hive
• Expanded Assessment Capabilities
• Data flows, query similarities, more recommendations
• Improved User Experience and Reporting

22. 22© Cloudera, Inc. All rights reserved.
HOLD FOR POLLING QUESTION
What is your need for a tool like this?
• Right now
• Next 1-6 months
• 6-12 months
• Greater than 12 months
• No need

23. 23© Cloudera, Inc. All rights reserved.
Getting Started
Cloudera Navigator Optimizer Now Available in Limited Beta
Request Beta Access at optimizer.cloudera.com

24. 24© Cloudera, Inc. All rights reserved.
Cloudera Enterprise
Making Hadoop Fast, Easy, and Secure
A new kind of data
platform:
• One place for unlimited data
• Unified, multi-framework data
access
Cloudera makes it:
• Fast for business
• Easy to manage
• Secure without compromise
OPERATE
GOVERN,
OPTIMIZE,
PROTECT
STRUCTURED UNSTRUCTURED
RESOURCE MANAGEMENT SECURITY
PROCESS, ANALYZE & SERVE
UNIFIED SERVICES
BATCH SQL STREAM SEARCH SDK
FILESYSTEM RELATIONAL NoSQL
STORE
INTEGRATE

25. 25© Cloudera, Inc. All rights reserved.
Thank You