If you’re a Denodo Partner, this presentation is for you. Learn how to gain a competitive edge in the marketplace with Denodo Platform 6.0, and leverage off the new features and functionality.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/Qh8MeX.
4. 4 Key Industry Data Trends
• Discovery and self-
service BI
• Access to data and
metadata in
business terms
• Canonical data
models
• Manage Queries across
large volumes of data
• New data types &
sources: Hadoop,
Spark,…
• Increased agility for
traditional BI, needs,
etc.
• SaaS
• Data
Distribution
• Multiple delivery
styles, formats
– REST, Odata,…
• Enterprise
Governance - Logical
data layer
• Lineage and change
impact
• Metadata and data
services sharing API
& catalog
V6.0
5. Denodo 6.0 Differentiation
New Query Rewritings
Statistics-based Data Movement
Incremental Cache
Support for new analytical data
sources
New Information Self-Service Tool
New ODBC Driver
Enhancements in Tableau
integration
Workload Management: Denodo
Resource Manager
Monitoring and Diagnosing Tool
Security & Governance
New VDP Admin Tool
GIT Support
BigData Connectivity
I. Performance in BigData Scenarios
II. Information Self-service
III. Management of Big Deployments
IV. Agile Development
V6.0
6. High Performance with Very Large Data
Volumes:
■ Logical Data Warehouse “Move Processing to the
Data Sources”
■ Full and partial aggregation pushdown
■ Statistics-based on-the-fly data movement
■ Cost-based automatic decision of the best
optimization strategy
■ Cache
■ Conditional Cache Requests: Caching +
incremental (delta) queries
■ Provides up-to-date cached data when queried
■ Support for new analytical data sources
V6.0
I. Performance in V6.0
7. V6.0
BigData Scenarios: Performance
New automatic query
rewritings to Push join
down below unions
Data Movement to
partitioned views
Example Technique:
Full Aggregation
Pushdown
8. When Data Lakes become “Data Swamps”
Big Data Queries Run Faster using DV
Denodo DV Query across
Impalas and Exadata Vs.
MDM and Large data sets in
Hadoop - Impala
ETL all data into Impala
and run full query there
MDM data in
Exadata
(Oracle)
Large Data sets in
Hadoop - Impala
Uncontrolled dumping of Data in Hadoop leads to poor perf. Solved by
• DV automatically collecting Statistics & Source capabilities, then
• Rewrites optimized queries and pushes processing down to the sources
• Thus, heavy processing is performed in the systems designed to do so:
• Impala Hadoop performs heavy aggregations on top of very large data sets
• Oracle Exadata is faster than Impala to process dimensional queries
9. V6.0
Big Data Queries Faster Using DV
Impala
Hadoop-only
Runtime (s)
Denodo
Runtime (s)
Denodo
Runtime w/
Cache (s)
Data Volumes
Query 1
199 120 68
Queries 1,2,3,5
•Exadata Row Count: ~5M
•Impala Row Count: ~500k
Query 4
•Exadata Row Count: ~5M
•Impala Row Count: ~2M
Query 2
187 96 88
Query 3
120 212 115
Query 4 timeout
328 69
Query 5
46 91 56
Performance comparison of 5 different queries :
Impala Hadoop versus DV across Impala & Exadata
DV delivers not only better or similar performance but Saves IT time &
resources to replicate the data into Hadoop
DV enables you to leverage Data Architectures for what they are good at.
10. V6.0
Logical Data Warehouse
Denodo DV Query across 3
data sources with Star schema
‘Sales’
Facts
(Netezza)
‘Customers’
Dimension
(Oracle)
‘Items’
Dimension
(SQL Server)290 M
2 M 400 K
Sales
(Netezza)
292.4 M
Vs.
ETL all data into Netezza
and run full query there
11. V6.0
Logical Data Warehouse Example
Physical equivalent to DV
Logical Performance
• DV again delivers similar
performance plus Saving IT
time & resources to replicate
the data into Netezza
• Automatically Selecting
Optimal Query Technique
Query Description
Returned
Rows
Avg. Time
Physical
(Netezza)
Denodo
Avg.
Time
Logical
Optimization
Technique
(automatically
chosen)
Total sales by
customer
1.99 M 21.0 sec 21. 5 sec
Full aggregation
push-down
Total sales by
customer and year
between 2000 and
2004
5.51 M 52.3 sec 59.1 sec
Full aggregation
push-down
Total sales by item
brand
31.4 K 4.7 sec 5.3 sec
Partial
aggregation push-
down
Total sales by item
where sale price
less than current
list price
17.1 K 3.5 sec 5.2 sec
On the fly data
movement
12. Before Rewriting
Problem
Join cannot be pushed down
Group By is not pushed
down
All sales sent to Reporting
Tools
Un-optimized Result
All Rows transferred: 100M
+ 10k
Slow execution and Netezza
is underutilized
J
G
100 M 10K
Sales
(Netezza)
Product
(Oracle)
SELECT p.id, p.name, SUM(s.amount)
as total
FROM product p join sales s ON
p.id=s.product_id
WHERE GETYEAR (s.sale_date) = 2015
GROUP BY p.id
Denodo Achieves High Perf. Using Proprietary
‘Dynamic Query Optimization Engine’
E.g. Query rewriting (Total sales by Product)
110.01M Rows
13. Benefit
Group By moved below JOIN
without affecting the results (PK-
FK join)
Group By pushed down to Netezza
Optimized Result
Rows transferred: 10K + 10k
DV enables x-source star-schema
features:
Size of Group By output determined
by cardinality of dimensions (small)
Star-schema joins allow Group By
push-down
J
10K
10K
Sales
(Netezza)
Product
(Oracle)
G
SELECT p.id, p.name, amount
FROM
(SELECT o.product_id, SUM (amount)
amount
FROM Order o
WHERE GETYEAR (s.sale_date) = 2015
GROUP BY o.product_id) og
JOIN Product p ON (p.id =
og.product_id)
Dynamic Query Optimization Engine
– Query rewriting (Total sales by Product)
After Rewriting
14. Cached Incremental Queries
Salesforce ‘Leads’ data
cached every 24 hours in
VDP at 7 AM before load gets
heavy
9 am Query needing last 24
hrs of ‘Leads’ (2 hours after
cache load)
New “Incremental Query”
combines 22 hrs from Cache
with 2 hr. Real-time from
SF.com
Response is up-to-date and
query is much faster since
smaller query sent to SF.com
during heavy load time.
Get Latest Leads Changed /
Added between 7 AM and 9
AM
CACHE
Leads updated
every 24
hours at 7 AM
V6.0
22 hours of
leads from
cache
2 hours of leads
real-time from
SF.com
Query at 9 AM:
get all Up-to-
date ‘Leads’ last
24 hrs
• Merge cached data and real-time (latest changes) data to provide
up-to-date results with minimum latency
• E.g. Makes queries to data sources in the cloud faster
15. V6.0
II. Information Discovery and Self-Service
New Data and Metadata Exploration Web Tool
for Business Users to:
■ Discover Metadata - available in their views including view
associations and lineage of every field
■ Find Relevant Data - perform global searches to all the views -
data and metadata
■ Create & Save Queries – of user favorite queries and publish them
as new view enabling reusability between users w/o writing SQL
New ODBC Driver – 50% faster
Enhancements Enable full Tableau integration
16. V6.0
Information Self Service Tool - I
Semantic Map - Access to a graphical model of business entities and
PK-FK associations
17. V6.0
Information Self Service Tool - II
Data Lineage - tree view information including derived fields
transformations
18. V6.0
Information Self Service Tool - III
GUI Based Query creation & save as new Denodo view
Export data via CSV, Tableau, Excel & HTML
20. V6.0
Information Self Service Tool - V
Global Search – enter keyword to find views containing that
data (create selective indexes for different users)
21. • Resource Manager - manages defined
Resource Plans
• Enhanced Monitoring & Diagnostic Tool
• Security & Governance
• Data Lineage/Tree View API
■ Integration point for CI tools and processes
v6.0
III. Management of Big Deployments
22. V6.0
IV. Agile Development 6.0
New Rapid Development Tool – less expertise needed
Automated Migration Dev to Prod.
E.g. Graphical Support for GIT as VCS System
Multiple Tabs
Multiple
Databases
23. V6.0
Continued Enhanced Connectivity
CRM
ERP
Warehouse
SaaS Apps
BigData
MarketingHuman
Resources
ETL
DATA VIRTUALIZATION
BigData and Cloud Databases Connectivity :
■ Redshift – enhanced adapter as data source, cache and data
movement target
■ Apache Spark – enhanced adapter
■ Vertica – enhanced as cache and data movement target
■ Impala – enhanced as cache and data movement target
26. Tools & Campaign Assets
• Business Assets
• Corporate PPT deck with 6.0 messaging
• Vertical specific PPT for Healthcare and Insurance
• Product brochure for 6.0
• Website content for 6.0
• Technical Assets
• Datasheet for Enterprise Platform 6.0
• Datasheet for Denodo Express 6.0
• Datasheet for Denodo Platform 6.0 on AWS
• Customer Support Guide (5.5 to 6.0)
• Lead Generation/Marketing Assets
• Marketing design elements/infographics
• Landing page
• HTML email invitation
• Unique registration URL
26
Denodo Platform 6.0
27. Marketing & Enablement
• Partner Portal
• Resource Centre
• Denodo Community
• Denodo Express
• Denodo Training
27
Resources
28. • Marketing
• Sales
• News & Events
• Training & Support
• Toolkits
Partner Portal
28
Exclusive portal that provides sales and marketing content to partners:
29. Toolkits
29
Downloadable self-service kits, designed to equip, educate and enable
Kickstarter Toolkit Marketing Toolkit Sales Engineering
Toolkit
Sales toolkit
Kickstart discussions, get on-boarded,
& equip yourself with the right tools.
Leverage the resources, templates,
and Denodo marketing engine.
Acquire the tools, know-how & skillset
to perform technical presentations.
Learn how to differentiate the
Denodo Platform when in the field.
30. • Brochures
• Case studies
• Datasheets
• Recorded demos
• E-books
• Podcasts
• Webinars
• Whitepapers
• Cookbooks
Denodo Resource Centre
30
Extensive library of data virtualization collateral and resources
31. • Q&A
• Knowledgebase
• Document library
• Video library
• Online tutorials
• Denodo Express
Denodo Community
31
Active community that provides technical support for customers and partners
32. • Training options
• Training curriculum
• Training categories
• Training paths
• Training packages/discounts
Denodo Training
32
Training options to suit all geographies and budgets
33. In Summary
33
Resource Source
Denodo Partner Portal partners.denodo.com
Denodo Resource Centre denodo.com/resources
Denodo Community community.denodo.com
Denodo Express community.denodo.com/express
Denodo Training denodo.com/services/education
34. Next Steps
34
Get Engaged
1. Join the Denodo Partner Program,
submit an application:
partners.denodo.com
2. Leverage the resources:
Partner Portal
Denodo Resource Centre
Denodo Community/Express
Denodo Training
3. Engage in joint co-marketing
activity or discuss an opportunity:
marketing.partners@denodo.com