Cloudera Federal Forum 2014: The Building Blocks of the Enterprise Data Hub

•

4 gefällt mir•2,566 views

Chief Technologist, Office of the CTO at Cloudera Eli Collins, shares the story of the enterprise data hub and how it relates to the enterprise data warehouse.

Technologie

Building Blocks of the Enterprise Data Hub
Eli Collins, Chief Technologist

1

©2014 Cloudera, Inc. All rights reserved.

The Enterprise Data Warehouse
Reporting

Operational
Store

Metadata

Operational
Store

Staging

Mining

Facts &
Dimensions

Summary

Analysis

Flat Files

Data Sources

2

EDW

©2014 Cloudera, Inc. All rights reserved.

Data Marts

The Enterprise Data Hub

MR, SQL, Spark, SAS, R, Search..

DB dumps

Logs
Images

3

1.
2.
3.
4.

Binar
y

©2014 Cloudera, Inc. All rights reserved.

Inexpensive storage
File system & KV storage
Co-located compute
Separate compute &
storage

So it’s Like a Data Warehouse?

4

©2014 Cloudera, Inc. All rights reserved.

An Analogy

5

©2014 Cloudera, Inc. All rights reserved.

What changed?
• The need?
• Convenience?

6

Cost?

©2014 Cloudera, Inc. All rights reserved.

Data Warehouse vs. Data Hub
Enterprise Data Warehouse

8

Enterprise Data Hub

©2014 Cloudera, Inc. All Rights Reserved.

An Operating System
App 1

App 2

App 3

File System
9

©2014 Cloudera, Inc. All Rights Reserved.

Services

Scheduler

Management

Lib

The Enterprise Data Hub

Search
Engine

Batch
Stream
Processing Processing

Machine
Learning

Resource Management

For Any Type of Data
Elastic, Fault-tolerant, Self-healing, In-memory capabilities

SQL

10

Streaming

File System (NFS)

©2014 Cloudera, Inc. All Rights Reserved.

Data
Management

Unified Scale-out Storage

System
Management

Analytic
MPP
DBMS

Metadata, Security, Audit, Lineage

Online
NoSQL
DBMS

11

©2014 Cloudera, Inc. All rights reserved.

12

©2014 Cloudera, Inc. All rights reserved.

13

©2014 Cloudera, Inc. All rights reserved.

14

©2014 Cloudera, Inc. All rights reserved.

Data Warehouse on the Data Hub
Connectors

MPP Query execution

ODBC/J
DBC

Impala
HBase/Accum
ulo

HDFS
Storage & formats

15

LLAMa

YARN
Resource Management

©2014 Cloudera, Inc. All Rights Reserved.

Hive
Metastore

Parquet, Avro,
Text, etc

UDFs
MadLib

Catalog

16

©2014 Cloudera, Inc. All rights reserved.

Empfohlen

Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet...ArabNet ME

The Future of Data Management: The Enterprise Data HubCloudera, Inc.

Better Together: The New Data Management OrchestraCloudera, Inc.

Enterprise Data Hub: The Next Big Thing in Big DataCloudera, Inc.

Govern This! Data Discovery and the application of data governance with new s...Cloudera, Inc.

Emergence of MongoDB as an Enterprise Data HubMongoDB

High-Performance Analytics in the Cloud with Apache ImpalaCloudera, Inc.

Hadoop: Extending your Data WarehouseCloudera, Inc.

Empfohlen

Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet...ArabNet ME

The Future of Data Management: The Enterprise Data HubCloudera, Inc.

Better Together: The New Data Management OrchestraCloudera, Inc.

Enterprise Data Hub: The Next Big Thing in Big DataCloudera, Inc.

Govern This! Data Discovery and the application of data governance with new s...Cloudera, Inc.

Emergence of MongoDB as an Enterprise Data HubMongoDB

High-Performance Analytics in the Cloud with Apache ImpalaCloudera, Inc.

Hadoop: Extending your Data WarehouseCloudera, Inc.

Optimized Data Management with Cloudera 5.7: Understanding data value with Cl...Cloudera, Inc.

MapR Enterprise Data Hub Webinar w/ Mike FergusonMapR Technologies

Building a Modern Analytic Database with Cloudera 5.8Cloudera, Inc.

Breakout: Data Discovery with HadoopCloudera, Inc.

How to Build Multi-disciplinary Analytics Applications on a Shared Data PlatformCloudera, Inc.

Part 1: Cloudera’s Analytic Database: BI & SQL Analytics in a Hybrid Cloud WorldCloudera, Inc.

Consolidate your data marts for fast, flexible analytics 5.24.18Cloudera, Inc.

Customer Best Practices: Optimizing Cloudera on AWSCloudera, Inc.

Part 1: Introducing the Cloudera Data Science WorkbenchCloudera, Inc.

The Transformation of your Data in modern IT (Presented by DellEMC)Cloudera, Inc.

Rethink Analytics with an Enterprise Data HubCloudera, Inc.

A beginners guide to Cloudera HadoopDavid Yahalom

Intuitive Real-Time Analytics with SearchCloudera, Inc.

It Takes a Village: Organizational Alignment to Deliver Big Data Value in Hea...DataWorks Summit

The Practice of Big Data - The Hadoop ecosystem explained with usage scenarioskcmallu

Driving Better Products with Customer Intelligence Cloudera, Inc.

DataStax on Azure: Deploying an industry-leading data platform for cloud apps...DataStax

Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...Cloudera, Inc.

Case study: Hadoop as ELT for Leading US Retailer - Happiest MindsHappiest Minds Technologies

Hadoop Essentials -- The What, Why and How to Meet Agency ObjectivesCloudera, Inc.

Justin Sheppard & Ankur Gupta from Sears Holdings Corporation - Single point ...Global Business Events

Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...MapR Technologies

Weitere ähnliche Inhalte

Was ist angesagt?

Optimized Data Management with Cloudera 5.7: Understanding data value with Cl...Cloudera, Inc.

MapR Enterprise Data Hub Webinar w/ Mike FergusonMapR Technologies

Building a Modern Analytic Database with Cloudera 5.8Cloudera, Inc.

Breakout: Data Discovery with HadoopCloudera, Inc.

How to Build Multi-disciplinary Analytics Applications on a Shared Data PlatformCloudera, Inc.

Part 1: Cloudera’s Analytic Database: BI & SQL Analytics in a Hybrid Cloud WorldCloudera, Inc.

Consolidate your data marts for fast, flexible analytics 5.24.18Cloudera, Inc.

Customer Best Practices: Optimizing Cloudera on AWSCloudera, Inc.

Part 1: Introducing the Cloudera Data Science WorkbenchCloudera, Inc.

The Transformation of your Data in modern IT (Presented by DellEMC)Cloudera, Inc.

Rethink Analytics with an Enterprise Data HubCloudera, Inc.

A beginners guide to Cloudera HadoopDavid Yahalom

Intuitive Real-Time Analytics with SearchCloudera, Inc.

It Takes a Village: Organizational Alignment to Deliver Big Data Value in Hea...DataWorks Summit

The Practice of Big Data - The Hadoop ecosystem explained with usage scenarioskcmallu

Driving Better Products with Customer Intelligence Cloudera, Inc.

DataStax on Azure: Deploying an industry-leading data platform for cloud apps...DataStax

Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...Cloudera, Inc.

Case study: Hadoop as ELT for Leading US Retailer - Happiest MindsHappiest Minds Technologies

Hadoop Essentials -- The What, Why and How to Meet Agency ObjectivesCloudera, Inc.

Was ist angesagt? (20)

Optimized Data Management with Cloudera 5.7: Understanding data value with Cl...

MapR Enterprise Data Hub Webinar w/ Mike Ferguson

Building a Modern Analytic Database with Cloudera 5.8

Breakout: Data Discovery with Hadoop

How to Build Multi-disciplinary Analytics Applications on a Shared Data Platform

Part 1: Cloudera’s Analytic Database: BI & SQL Analytics in a Hybrid Cloud World

Consolidate your data marts for fast, flexible analytics 5.24.18

Customer Best Practices: Optimizing Cloudera on AWS

Part 1: Introducing the Cloudera Data Science Workbench

The Transformation of your Data in modern IT (Presented by DellEMC)

Rethink Analytics with an Enterprise Data Hub

A beginners guide to Cloudera Hadoop

Intuitive Real-Time Analytics with Search

It Takes a Village: Organizational Alignment to Deliver Big Data Value in Hea...

The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios

Driving Better Products with Customer Intelligence 

DataStax on Azure: Deploying an industry-leading data platform for cloud apps...

Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...

Case study: Hadoop as ELT for Leading US Retailer - Happiest Minds

Hadoop Essentials -- The What, Why and How to Meet Agency Objectives

Andere mochten auch

Justin Sheppard & Ankur Gupta from Sears Holdings Corporation - Single point ...Global Business Events

Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...MapR Technologies

Standing Up an Effective Enterprise Data Hub -- Technology and BeyondCloudera, Inc.

The Future of Data Management: The Enterprise Data HubCloudera, Inc.

Apache HBase in the Enterprise Data Hub at CernerHBaseCon

Big Data Testing: Ensuring MongoDB Data QualityRTTS

Testing Big Data: Automated Testing of Hadoop with QuerySurgeRTTS

Andere mochten auch (7)

Justin Sheppard & Ankur Gupta from Sears Holdings Corporation - Single point ...

Data on the Move: Transitioning from a Legacy Architecture to a Big Data Plat...

Standing Up an Effective Enterprise Data Hub -- Technology and Beyond

The Future of Data Management: The Enterprise Data Hub

Apache HBase in the Enterprise Data Hub at Cerner

Big Data Testing: Ensuring MongoDB Data Quality

Testing Big Data: Automated Testing of Hadoop with QuerySurge

Ähnlich wie Cloudera Federal Forum 2014: The Building Blocks of the Enterprise Data Hub

Syncsort, Tableau, & Cloudera present: Break the Barriers to Big Data InsightSteven Totman

Syncsort, Tableau, & Cloudera present: Break the Barriers to Big Data InsightCloudera, Inc.

Syncsort, Tableau, & Cloudera present: Break the Barriers to Big Data InsightPrecisely

Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and ...jdijcks

Simplifying Real-Time Architectures for IoT with Apache KuduCloudera, Inc.

Oracle Big Data Appliance and Big Data SQL for advanced analyticsjdijcks

Hadoop and ManufacturingCloudera, Inc.

Part 2: Apache Kudu: Extending the Capabilities of Operational and Analytic D...Cloudera, Inc.

Breakout: Hadoop and the Operational Data StoreCloudera, Inc.

Big Data LDN 2018: MICROSOFT AZURE AND CLOUDERA – FLEXIBLE CLOUD, WHATEVER TH...Matt Stubbs

Supply chain analyticsNitai Partners Inc

Pivotal the new_pivotal_big_data_suite_-_revolutionary_foundation_to_leverage...EMC

Google take on heterogeneous data base replication Svetlin Stanchev

Big Data: Myths and RealitiesToronto-Oracle-Users-Group

Making Hadoop based analytics simple for everyone to useSwiss Big Data User Group

Cloudera Federal Forum 2014: Cloud Deployment for the Enterprise Data HubCloudera, Inc.

Hr and performance analyticsNitai Partners Inc

Idera live 2021: Managing Digital Transformation on a Budget by Bert ScalzoIDERA Software

Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platformEMC

Enterprise asset management analyticsNitai Partners Inc

Ähnlich wie Cloudera Federal Forum 2014: The Building Blocks of the Enterprise Data Hub (20)

Syncsort, Tableau, & Cloudera present: Break the Barriers to Big Data Insight

Oracle Openworld Presentation with Paul Kent (SAS) on Big Data Appliance and ...

Simplifying Real-Time Architectures for IoT with Apache Kudu

Oracle Big Data Appliance and Big Data SQL for advanced analytics

Hadoop and Manufacturing

Part 2: Apache Kudu: Extending the Capabilities of Operational and Analytic D...

Breakout: Hadoop and the Operational Data Store

Big Data LDN 2018: MICROSOFT AZURE AND CLOUDERA – FLEXIBLE CLOUD, WHATEVER TH...

Supply chain analytics

Pivotal the new_pivotal_big_data_suite_-_revolutionary_foundation_to_leverage...

Google take on heterogeneous data base replication

Big Data: Myths and Realities

Making Hadoop based analytics simple for everyone to use

Cloudera Federal Forum 2014: Cloud Deployment for the Enterprise Data Hub

Hr and performance analytics

Idera live 2021: Managing Digital Transformation on a Budget by Bert Scalzo

Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform

Enterprise asset management analytics

Mehr von Cloudera, Inc.

Partner Briefing_January 25 (FINAL).pptxCloudera, Inc.

Cloudera Data Impact Awards 2021 - Finalists Cloudera, Inc.

2020 Cloudera Data Impact Awards FinalistsCloudera, Inc.

Edc event vienna presentation 1 oct 2019Cloudera, Inc.

Machine Learning with Limited Labeled Data 4/3/19Cloudera, Inc.

Data Driven With the Cloudera Modern Data Warehouse 3.19.19Cloudera, Inc.

Introducing Cloudera DataFlow (CDF) 2.13.19Cloudera, Inc.

Introducing Cloudera Data Science Workbench for HDP 2.12.19Cloudera, Inc.

Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Cloudera, Inc.

Leveraging the cloud for analytics and machine learning 1.29.19Cloudera, Inc.

Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Cloudera, Inc.

Leveraging the Cloud for Big Data Analytics 12.11.18Cloudera, Inc.

Modern Data Warehouse Fundamentals Part 3Cloudera, Inc.

Modern Data Warehouse Fundamentals Part 2Cloudera, Inc.

Modern Data Warehouse Fundamentals Part 1Cloudera, Inc.

Extending Cloudera SDX beyond the PlatformCloudera, Inc.

Federated Learning: ML with Privacy on the Edge 11.15.18Cloudera, Inc.

Analyst Webinar: Doing a 180 on Customer 360Cloudera, Inc.

Build a modern platform for anti-money laundering 9.19.18Cloudera, Inc.

Introducing the data science sandbox as a service 8.30.18Cloudera, Inc.

Mehr von Cloudera, Inc. (20)

Partner Briefing_January 25 (FINAL).pptx

Cloudera Data Impact Awards 2021 - Finalists

2020 Cloudera Data Impact Awards Finalists

Edc event vienna presentation 1 oct 2019

Machine Learning with Limited Labeled Data 4/3/19

Data Driven With the Cloudera Modern Data Warehouse 3.19.19

Introducing Cloudera DataFlow (CDF) 2.13.19

Introducing Cloudera Data Science Workbench for HDP 2.12.19

Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19

Leveraging the cloud for analytics and machine learning 1.29.19

Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19

Leveraging the Cloud for Big Data Analytics 12.11.18

Modern Data Warehouse Fundamentals Part 3

Modern Data Warehouse Fundamentals Part 2

Modern Data Warehouse Fundamentals Part 1

Extending Cloudera SDX beyond the Platform

Federated Learning: ML with Privacy on the Edge 11.15.18

Analyst Webinar: Doing a 180 on Customer 360

Build a modern platform for anti-money laundering 9.19.18

Introducing the data science sandbox as a service 8.30.18

Kürzlich hochgeladen

Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani

Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan

Manual 508 Accessibility Compliance AuditSkynet Technologies

How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes

Connecting the Dots for Information Discovery.pdfNeo4j

Time Series Foundation Models - current state and future directionsNathaniel Shimoni

Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes

From Family Reminiscence to Scholarly Archive .Alan Dix

Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3

The State of Passkeys with FIDO Alliance.pptxLoriGlavin3

The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech

Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González

UiPath Community: Communication Mining from Zero to HeroUiPathCommunity

TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc

Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda

Sample pptx for embedding into website for demoHarshalMandlekar2

Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll

Rise of the Machines: Known As Drones...Rick Flair

Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3

Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3

Kürzlich hochgeladen (20)

Potential of AI (Generative AI) in Business: Learnings and Insights

Generative AI for Technical Writer or Information Developers

Manual 508 Accessibility Compliance Audit

How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes

Connecting the Dots for Information Discovery.pdf

Time Series Foundation Models - current state and future directions

Assure Ecommerce and Retail Operations Uptime with ThousandEyes

From Family Reminiscence to Scholarly Archive .

Moving Beyond Passwords: FIDO Paris Seminar.pdf

The State of Passkeys with FIDO Alliance.pptx

The Ultimate Guide to Choosing WordPress Pros and Cons

Generative Artificial Intelligence: How generative AI works.pdf

UiPath Community: Communication Mining from Zero to Hero

TrustArc Webinar - How to Build Consumer Trust Through Data Privacy

Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...

Sample pptx for embedding into website for demo

Emixa Mendix Meetup 11 April 2024 about Mendix Native development

Rise of the Machines: Known As Drones...

Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx

Digital Identity is Under Attack: FIDO Paris Seminar.pptx

Cloudera Federal Forum 2014: The Building Blocks of the Enterprise Data Hub

2. The Enterprise Data Warehouse Reporting Operational Store Metadata Operational Store Staging Mining Facts & Dimensions Summary Analysis Flat Files Data Sources 2 EDW ©2014 Cloudera, Inc. All rights reserved. Data Marts

3. The Enterprise Data Hub MR, SQL, Spark, SAS, R, Search.. DB dumps Logs Images 3 1. 2. 3. 4. Binar y ©2014 Cloudera, Inc. All rights reserved. Inexpensive storage File system & KV storage Co-located compute Separate compute & storage

7. Take and share good photos 7

10. The Enterprise Data Hub Search Engine Batch Stream Processing Processing Machine Learning Resource Management For Any Type of Data Elastic, Fault-tolerant, Self-healing, In-memory capabilities SQL 10 Streaming File System (NFS) ©2014 Cloudera, Inc. All Rights Reserved. Data Management Unified Scale-out Storage System Management Analytic MPP DBMS Metadata, Security, Audit, Lineage Online NoSQL DBMS

15. Data Warehouse on the Data Hub Connectors MPP Query execution ODBC/J DBC Impala HBase/Accum ulo HDFS Storage & formats 15 LLAMa YARN Resource Management ©2014 Cloudera, Inc. All Rights Reserved. Hive Metastore Parquet, Avro, Text, etc UDFs MadLib Catalog