SlideShare ist ein Scribd-Unternehmen logo
1 von 22
Downloaden Sie, um offline zu lesen
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HPHAVEn
BigDataUseCases
Mikolaj Nietz, Solution Architect
Application Services Global Delivery,
Hewlett-Packard
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The changing Big Data landscape
Human InformationMachine Data
Business
Data
10% of Information
90% of Information
Annual
Growth
~100%
~10%
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Interact with and process 100% of your data seamlessly
Imagine if you could…
Transactional
data Social media Images AudioVideoMobile Email TextsDocumentsIn-memoryHadoop
Standard APIs and tools
Dashboards & alerts Business intelligence Your custom appsPackaged apps
Ingest Analyze Understand
Machine Data Business Data Human Information
Open connectors
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data Platform
HAVEn
HAVEn
Social media IT/OT ImagesAudioVideo Transactional
data
Mobile Search engineEmail Texts
Catalogue massive
volumes of
distributed data
Hadoop/
HDFS
Process and index
all information
Autonomy
IDOL
Analyze at
extreme scale
in real-time
Vertica
Collect & unify
machine data
Enterprise
Security
Powering
HP Software
+ your apps
nApps
Documents
hp.com/Haven
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Why HAVEn?
Hadoop
Autonomy IDOL
Vertica
Enterprise Security (HP ArcSight)
n – a numer of other apps
„Safe Haven” = „Bezpieczna Przystań”
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP HAVEn/Big Data
Reference Architecture
Rich-media data
Unstructured
text data
Mixed-structure
data
Unknown-structure
data
Semi-structured
text data
Structured
text data
ODS
EDW
Data marts
Hadoop
HDFS
Map Reduce
Data integration
NotOnly SQL
Analytics
Operational mgt.
Access-in-place
Meaning-based
analytics
(Autonomy IDOL)
Autonomy
value-add
applications
BI/
Visualization
tools
Analytic
tools
Lightweight
ETL
Hadoop Extended Tools
Access-in-place
Indexed metadata
Vertica
Analytics RDBMS
Native analytics
UDx extensions
R-Functions
Access-in-place
Indexed metadata
WWW
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Apache Hadoop
Has flexibility to store and mine
any type of data
• Query previously inaccessible
structured and unstructured data
• Not bound by single schema
Excels at processing
complex data
• Scale-out architecture divides
workloads across multiple nodes
• Flexible file system eliminates
ETL* bottlenecks
Scales
economically
• Deployable on commodity
hardware
• Open source platform guards
against vendor lock
Hadoop
Distributed File
System (HDFS)
Self-healing,
high bandwidth
clustered storage
MapReduce
Distributed
Computing
Framework
Open source Linux-based platform for
data storage and processing that is…
 Scalable
 Fault tolerant
 Distributed
Core HADOOP system components (Workloads)
Like Linux, there are several distributions of Hadoop
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Autonomy IDOL
Social Media Video Audio Email Texts Mobile Transactional
Data
Documents XML Search Engine Images
HP Autonomy
IDOL Applications
Autonomy Connectors
eDiscovery
Enterprise Search
Media
Monitoring
Social Media
Analytics
Decision
Support
Augmented
Reality
Partner/
In-house apps
HC Analytics
Repositories
Information
Types
Apps
500
Functions
IDOL Services Multimedia
Informatics
Enrichment
Capture
InteractionAnalytics
Discovery
Concept
Clouds
Active
MatchingVisualization
ACA
MediaBin
Connected LiveVault
TRIM
AeD
Data Protector
WorkSite
DigitalSafe
Connectors
…
CloudEnterprise
IDOL
OS for Human Information
ERP
CRM
Database Jive…
Image
HIS
Data Warehouse
Hadoop
SharePoint
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Seamlessly access virtually any enterprise content repository, including file systems, email, or
knowledge bases
400+ connectors
All data types, all content repositories – unmatched understanding
HP Autonomy IDOL platform
High-performance human information processing
HP Autonomy IDOL
Leverage the power of functions like sentiment, categorization, and clustering to deliver intelligence and
insight
Over 500 functions
Process virtually any file type such as text (email, tweet, document), audio, video, and even people
profiles & behavior
1,000+ file types
Achieve big data scalability and high performance with distributable ingest and query architecture
Distributable architecture
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Vertica
Real Time Analytics Platform
Standard SQL Interface
Native
High
Availability
Auto
Database
Design
Advanced
Compression
Column
Orientation
MPP Massively
Parallel
Processing
Leverages BI, ETL,
Hadoop/MapReduce and
OLTP investments
Automatic setup,
optimization, and
DB management
Built-in redundancy
that also speeds up
queries
Native DB-aware
clustering on low-cost
x86 Linux nodes
Up to 90% space
reduction using 12+
algorithms
• 10x – 100x performance than
classic RDBMS
• High scalability from TBs to
PBs
• Simple integration with
existing ETL and BI solutions
• Superior performance on off-
the-shelf hardware
• Ultimate deployment flexibility
• 24/7 Load and Query
• Flexzone
• Very close Hadoop integration
• Soon-to-come: Vertica-on-
Yarn
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Why Hadoop and Vertica are complementary
• Designed for Performance
• Interactive Analytics
• A Rich SQL Ecosystem
• Designed for Fault Tolerance
• Storage & Batch Processing
• A Rich Programming Model
Both purpose-built scalable platforms
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Gain insight into your data in near-real time by running queries 50x-1,000x faster than legacy products
Blazing fast analytics
Speed, scalability, and openness at lower TCO
HP Vertica Analytics platform
High-performance data analytics platform purpose-built for big data
HP Vertica
Infinitely scale your solution by adding an unlimited number of industry-standard servers
Massive scalability
Protect and embrace your investment in hardware and software with built-in support for Hadoop, R, and
a range of ETL and BI tools
Open architecture
Store 10x-30x more data per server than row databases with patented columnar compression
Optimized data storage
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Collect, normalize, and categorize machine data such as logs, events, and flows from any device, any
time, anywhere from any vendor
315+ connectors
Collect, store, and analyze any machine data across IT
HP ArcSight Universal log management platform
High-performance universal log management to consolidate machine data across IT
HP ArcSight
The unified machine data through filtering and parsing is enriched with rich metadata, which allows you to search
machine data through simple text-based keywords without the need of domain expertise
Search over 1,000,000 events per second
The unified data is stored through high compression ratio in any of your existing storage formats,
eliminating the need for expensive databases and DBAs
Store years’ worth of data
Built-in content packs, algorithms, rules, and the unified machine data help you deploy IT
security, IT operations, IT GRC, and log analytics
Analytics & intelligence
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The „n”
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Autonomy + Vertica + Tableau + HP Anywhere on Tablet
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
German Car Manufacturer
Early Warning System
Business problem
Detect unusual increases in the number of
warranty repairs (OT warranty) as soon as they
appear.
Data analysis problem
Detect anomalies (outliers) in time series.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
External
Internal
German Car Manufacturer
Big Data Labs
Warranty
Repairs
Landing
Zone
Integrated
Data
Analytical
Record
Analytical
Processing
Visualization
HP HAVEn Platform
Repairs
Claims
Sales
Storage
Parts &
Production
Diagnostics
Reference
Weather
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Global Telecommunication Group
Log Analysis
Vertica ClusterNFS
Hadoop Cluster
Log System
POC environment
Vertica Hadoop Connector
JDBC
3 Vertica nodes:
• 2x2 core Intel XEON @ 2.7 GHz
• 32 GB RAM
• 9.7 TB storage
Java applications
Analytics & Reporting
clients
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Global Cranes Manufacturer
Sensor Data Analysis
Remote
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Facebook
Big Data Architecture for Log Analysis
Mobile
PC/Laptop
Web Servers
Logs
Hadoop/
HDFS 2 huge Hadoop
Clusters
• 1.7 ExaBytes
• 15000 nodes
• 40000 nodes
Job
Scheduler
Vertica
Logs
15
mins
Hourly
Daily
Legacy
• 600K MR Jobs/day
• 50K Informatica Jobs/day
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Develop Operate
SecureMonetize
Govern
HAVEn
hp.com/haven
Thank you!
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Resources:
• www.hp.comhaven
• www.vertica.com
• www.autonomy.com
• www.hortonworks.com
• Vertica to try:
https://my.vertica.com/?redirect_to=https%3A%2F%2Fmy.vertica.com%2Fdownl
oad-community-edition%2F
• About HAVEn-on-demand:
http://www.datacenterknowledge.com/archives/2014/12/03/hp-launches-big-
data-cloud-called-haven-ondemand/

Weitere ähnliche Inhalte

Was ist angesagt?

Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...
Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...
Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...mustafa sarac
 
Optimize your cloud strategy for machine learning and analytics
Optimize your cloud strategy for machine learning and analyticsOptimize your cloud strategy for machine learning and analytics
Optimize your cloud strategy for machine learning and analyticsCloudera, Inc.
 
Informatica Becomes Part of the Business Data Lake Ecosystem
Informatica Becomes Part of the Business Data Lake EcosystemInformatica Becomes Part of the Business Data Lake Ecosystem
Informatica Becomes Part of the Business Data Lake EcosystemCapgemini
 
Benefiting from Big Data - A New Approach for the Telecom Industry
Benefiting from Big Data - A New Approach for the Telecom Industry  Benefiting from Big Data - A New Approach for the Telecom Industry
Benefiting from Big Data - A New Approach for the Telecom Industry Persontyle
 
Cox Automotive: data sells cars
Cox Automotive: data sells carsCox Automotive: data sells cars
Cox Automotive: data sells carsCloudera, Inc.
 
Big data analytics for telecom operators final use cases 0712-2014_prof_m erdas
Big data analytics for telecom operators final use cases 0712-2014_prof_m erdasBig data analytics for telecom operators final use cases 0712-2014_prof_m erdas
Big data analytics for telecom operators final use cases 0712-2014_prof_m erdasProf Dr Mehmed ERDAS
 
The Big Picture: Real-time Data is Defining Intelligent Offers
The Big Picture: Real-time Data is Defining Intelligent OffersThe Big Picture: Real-time Data is Defining Intelligent Offers
The Big Picture: Real-time Data is Defining Intelligent OffersCloudera, Inc.
 
Telco Big Data 2012 Highlights
Telco Big Data 2012 HighlightsTelco Big Data 2012 Highlights
Telco Big Data 2012 HighlightsAlan Quayle
 
Strategyzing big data in telco industry
Strategyzing big data in telco industryStrategyzing big data in telco industry
Strategyzing big data in telco industryParviz Iskhakov
 
Customer Experience: A Catalyst for Digital Transformation
Customer Experience: A Catalyst for Digital TransformationCustomer Experience: A Catalyst for Digital Transformation
Customer Experience: A Catalyst for Digital TransformationCloudera, Inc.
 
Monetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service ProvidersMonetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service ProvidersCubic Corporation
 
Big Data and Analytics: The IBM Perspective
Big Data and Analytics: The IBM PerspectiveBig Data and Analytics: The IBM Perspective
Big Data and Analytics: The IBM PerspectiveThe_IPA
 
WCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEn
WCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEnWCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEn
WCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEnWCIT 2014
 
Big Data Predictions for 2015
Big Data Predictions for 2015 Big Data Predictions for 2015
Big Data Predictions for 2015 Pentaho
 
HPE IDOL Technical Overview - july 2016
HPE IDOL Technical Overview - july 2016HPE IDOL Technical Overview - july 2016
HPE IDOL Technical Overview - july 2016Andrey Karpov
 
The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...
The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...
The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...Datameer
 
Big Data Expo 2015 - Pentaho The Future of Analytics
Big Data Expo 2015 - Pentaho The Future of AnalyticsBig Data Expo 2015 - Pentaho The Future of Analytics
Big Data Expo 2015 - Pentaho The Future of AnalyticsBigDataExpo
 
Big Data Case study - caixa bank
Big Data Case study - caixa bankBig Data Case study - caixa bank
Big Data Case study - caixa bankChungsik Yun
 
Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...
Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...
Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...Impetus Technologies
 

Was ist angesagt? (20)

Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...
Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...
Big data & advanced analytics in Telecom: A multi-billion-dollar revenue oppo...
 
Optimize your cloud strategy for machine learning and analytics
Optimize your cloud strategy for machine learning and analyticsOptimize your cloud strategy for machine learning and analytics
Optimize your cloud strategy for machine learning and analytics
 
Informatica Becomes Part of the Business Data Lake Ecosystem
Informatica Becomes Part of the Business Data Lake EcosystemInformatica Becomes Part of the Business Data Lake Ecosystem
Informatica Becomes Part of the Business Data Lake Ecosystem
 
Benefiting from Big Data - A New Approach for the Telecom Industry
Benefiting from Big Data - A New Approach for the Telecom Industry  Benefiting from Big Data - A New Approach for the Telecom Industry
Benefiting from Big Data - A New Approach for the Telecom Industry
 
Big Data use cases in telcos
Big Data use cases in telcosBig Data use cases in telcos
Big Data use cases in telcos
 
Cox Automotive: data sells cars
Cox Automotive: data sells carsCox Automotive: data sells cars
Cox Automotive: data sells cars
 
Big data analytics for telecom operators final use cases 0712-2014_prof_m erdas
Big data analytics for telecom operators final use cases 0712-2014_prof_m erdasBig data analytics for telecom operators final use cases 0712-2014_prof_m erdas
Big data analytics for telecom operators final use cases 0712-2014_prof_m erdas
 
The Big Picture: Real-time Data is Defining Intelligent Offers
The Big Picture: Real-time Data is Defining Intelligent OffersThe Big Picture: Real-time Data is Defining Intelligent Offers
The Big Picture: Real-time Data is Defining Intelligent Offers
 
Telco Big Data 2012 Highlights
Telco Big Data 2012 HighlightsTelco Big Data 2012 Highlights
Telco Big Data 2012 Highlights
 
Strategyzing big data in telco industry
Strategyzing big data in telco industryStrategyzing big data in telco industry
Strategyzing big data in telco industry
 
Customer Experience: A Catalyst for Digital Transformation
Customer Experience: A Catalyst for Digital TransformationCustomer Experience: A Catalyst for Digital Transformation
Customer Experience: A Catalyst for Digital Transformation
 
Monetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service ProvidersMonetizing Big Data with Streaming Analytics for Telecoms Service Providers
Monetizing Big Data with Streaming Analytics for Telecoms Service Providers
 
Big Data and Analytics: The IBM Perspective
Big Data and Analytics: The IBM PerspectiveBig Data and Analytics: The IBM Perspective
Big Data and Analytics: The IBM Perspective
 
WCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEn
WCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEnWCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEn
WCIT 2014 Rohit Tandon - Big Data to Drive Business Results: HP HAVEn
 
Big Data Predictions for 2015
Big Data Predictions for 2015 Big Data Predictions for 2015
Big Data Predictions for 2015
 
HPE IDOL Technical Overview - july 2016
HPE IDOL Technical Overview - july 2016HPE IDOL Technical Overview - july 2016
HPE IDOL Technical Overview - july 2016
 
The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...
The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...
The State of Big Data Adoption: A Glance at Top Industries Adopting Big Data ...
 
Big Data Expo 2015 - Pentaho The Future of Analytics
Big Data Expo 2015 - Pentaho The Future of AnalyticsBig Data Expo 2015 - Pentaho The Future of Analytics
Big Data Expo 2015 - Pentaho The Future of Analytics
 
Big Data Case study - caixa bank
Big Data Case study - caixa bankBig Data Case study - caixa bank
Big Data Case study - caixa bank
 
Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...
Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...
Big Data Use Cases for Different Verticals and Adoption Patterns - Impetus We...
 

Ähnlich wie Haven 2 0

A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...DataWorks Summit
 
4. Big data & analytics HP
4. Big data & analytics HP4. Big data & analytics HP
4. Big data & analytics HPMITEF México
 
IoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJIoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJDaniel Madrigal
 
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for AnalyticsVerizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for AnalyticsDataWorks Summit
 
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...Hortonworks
 
Carpe Datum: Building Big Data Analytical Applications with HP Haven
Carpe Datum: Building Big Data Analytical Applications with HP HavenCarpe Datum: Building Big Data Analytical Applications with HP Haven
Carpe Datum: Building Big Data Analytical Applications with HP HavenDataWorks Summit
 
Transform Your Business with Big Data and Hortonworks
Transform Your Business with Big Data and Hortonworks Transform Your Business with Big Data and Hortonworks
Transform Your Business with Big Data and Hortonworks Pactera_US
 
Transform You Business with Big Data and Hortonworks
Transform You Business with Big Data and HortonworksTransform You Business with Big Data and Hortonworks
Transform You Business with Big Data and HortonworksHortonworks
 
Exclusive Verizon Employee Webinar: Getting More From Your CDR Data
Exclusive Verizon Employee Webinar: Getting More From Your CDR DataExclusive Verizon Employee Webinar: Getting More From Your CDR Data
Exclusive Verizon Employee Webinar: Getting More From Your CDR DataPentaho
 
Create a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache HadoopCreate a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache HadoopHortonworks
 
To Serve and Protect: Making Sense of Hadoop Security
To Serve and Protect: Making Sense of Hadoop Security To Serve and Protect: Making Sense of Hadoop Security
To Serve and Protect: Making Sense of Hadoop Security Inside Analysis
 
Hadoop Reporting and Analysis - Jaspersoft
Hadoop Reporting and Analysis - JaspersoftHadoop Reporting and Analysis - Jaspersoft
Hadoop Reporting and Analysis - JaspersoftHortonworks
 
Hadoop: Extending your Data Warehouse
Hadoop: Extending your Data WarehouseHadoop: Extending your Data Warehouse
Hadoop: Extending your Data WarehouseCloudera, Inc.
 
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...Hortonworks
 
Big data tim
Big data timBig data tim
Big data timT Weir
 
Hortonworks and Voltage Security webinar
Hortonworks and Voltage Security webinarHortonworks and Voltage Security webinar
Hortonworks and Voltage Security webinarHortonworks
 
Supporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big DataSupporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big DataWANdisco Plc
 
Hortonworks and Red Hat Webinar_Sept.3rd_Part 1
Hortonworks and Red Hat Webinar_Sept.3rd_Part 1Hortonworks and Red Hat Webinar_Sept.3rd_Part 1
Hortonworks and Red Hat Webinar_Sept.3rd_Part 1Hortonworks
 

Ähnlich wie Haven 2 0 (20)

A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...A modern, flexible approach to Hadoop implementation incorporating innovation...
A modern, flexible approach to Hadoop implementation incorporating innovation...
 
4. Big data & analytics HP
4. Big data & analytics HP4. Big data & analytics HP
4. Big data & analytics HP
 
Solving Big Data Problems using Hortonworks
Solving Big Data Problems using Hortonworks Solving Big Data Problems using Hortonworks
Solving Big Data Problems using Hortonworks
 
IoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJIoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJ
 
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for AnalyticsVerizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
 
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
 
Carpe Datum: Building Big Data Analytical Applications with HP Haven
Carpe Datum: Building Big Data Analytical Applications with HP HavenCarpe Datum: Building Big Data Analytical Applications with HP Haven
Carpe Datum: Building Big Data Analytical Applications with HP Haven
 
Transform Your Business with Big Data and Hortonworks
Transform Your Business with Big Data and Hortonworks Transform Your Business with Big Data and Hortonworks
Transform Your Business with Big Data and Hortonworks
 
Transform You Business with Big Data and Hortonworks
Transform You Business with Big Data and HortonworksTransform You Business with Big Data and Hortonworks
Transform You Business with Big Data and Hortonworks
 
Exclusive Verizon Employee Webinar: Getting More From Your CDR Data
Exclusive Verizon Employee Webinar: Getting More From Your CDR DataExclusive Verizon Employee Webinar: Getting More From Your CDR Data
Exclusive Verizon Employee Webinar: Getting More From Your CDR Data
 
Create a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache HadoopCreate a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache Hadoop
 
To Serve and Protect: Making Sense of Hadoop Security
To Serve and Protect: Making Sense of Hadoop Security To Serve and Protect: Making Sense of Hadoop Security
To Serve and Protect: Making Sense of Hadoop Security
 
Hadoop Reporting and Analysis - Jaspersoft
Hadoop Reporting and Analysis - JaspersoftHadoop Reporting and Analysis - Jaspersoft
Hadoop Reporting and Analysis - Jaspersoft
 
Hadoop: Extending your Data Warehouse
Hadoop: Extending your Data WarehouseHadoop: Extending your Data Warehouse
Hadoop: Extending your Data Warehouse
 
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
 
Big data tim
Big data timBig data tim
Big data tim
 
Hortonworks and Voltage Security webinar
Hortonworks and Voltage Security webinarHortonworks and Voltage Security webinar
Hortonworks and Voltage Security webinar
 
Supporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big DataSupporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big Data
 
Hortonworks and Red Hat Webinar_Sept.3rd_Part 1
Hortonworks and Red Hat Webinar_Sept.3rd_Part 1Hortonworks and Red Hat Webinar_Sept.3rd_Part 1
Hortonworks and Red Hat Webinar_Sept.3rd_Part 1
 
xGem BigData
xGem BigDataxGem BigData
xGem BigData
 

Mehr von Data Science Warsaw

Wizualne budowanie aplikacji na Sparku przy pomocy narzędzia Seahorse
Wizualne budowanie aplikacji na Sparku przy pomocy narzędzia SeahorseWizualne budowanie aplikacji na Sparku przy pomocy narzędzia Seahorse
Wizualne budowanie aplikacji na Sparku przy pomocy narzędzia SeahorseData Science Warsaw
 
Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...
Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...
Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...Data Science Warsaw
 
CRISP-DM Agile Approach to Data Mining Projects
CRISP-DM Agile Approach to Data Mining ProjectsCRISP-DM Agile Approach to Data Mining Projects
CRISP-DM Agile Approach to Data Mining ProjectsData Science Warsaw
 
Online content popularity prediction
Online content popularity predictionOnline content popularity prediction
Online content popularity predictionData Science Warsaw
 
Rozwiązywanie problemów optymalizacyjnych
Rozwiązywanie problemów optymalizacyjnychRozwiązywanie problemów optymalizacyjnych
Rozwiązywanie problemów optymalizacyjnychData Science Warsaw
 
Big Data, Wearable, sztuczna inteligencja i ekonomia współpracy
Big  Data, Wearable, sztuczna inteligencja i ekonomia współpracyBig  Data, Wearable, sztuczna inteligencja i ekonomia współpracy
Big Data, Wearable, sztuczna inteligencja i ekonomia współpracyData Science Warsaw
 
Oracle Big Data Discovery - ludzka twarz Hadoop'a
Oracle Big Data Discovery - ludzka twarz Hadoop'aOracle Big Data Discovery - ludzka twarz Hadoop'a
Oracle Big Data Discovery - ludzka twarz Hadoop'aData Science Warsaw
 
Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!
Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!
Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!Data Science Warsaw
 
Data Exchange - the missing link in the big data value chain
Data Exchange - the missing link in the big data value chainData Exchange - the missing link in the big data value chain
Data Exchange - the missing link in the big data value chainData Science Warsaw
 
Metody logiczne w analizie danych
Metody logiczne w analizie danych Metody logiczne w analizie danych
Metody logiczne w analizie danych Data Science Warsaw
 

Mehr von Data Science Warsaw (20)

Wizualne budowanie aplikacji na Sparku przy pomocy narzędzia Seahorse
Wizualne budowanie aplikacji na Sparku przy pomocy narzędzia SeahorseWizualne budowanie aplikacji na Sparku przy pomocy narzędzia Seahorse
Wizualne budowanie aplikacji na Sparku przy pomocy narzędzia Seahorse
 
Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...
Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...
Neptune - narzędzie do monitorowania i zarządzania eksperymentami Machine Lea...
 
CRISP-DM Agile Approach to Data Mining Projects
CRISP-DM Agile Approach to Data Mining ProjectsCRISP-DM Agile Approach to Data Mining Projects
CRISP-DM Agile Approach to Data Mining Projects
 
Online content popularity prediction
Online content popularity predictionOnline content popularity prediction
Online content popularity prediction
 
Rozwiązywanie problemów optymalizacyjnych
Rozwiązywanie problemów optymalizacyjnychRozwiązywanie problemów optymalizacyjnych
Rozwiązywanie problemów optymalizacyjnych
 
Ile informacji jest w danych?
Ile informacji jest w danych?Ile informacji jest w danych?
Ile informacji jest w danych?
 
Analiza języka naturalnego
Analiza języka naturalnegoAnaliza języka naturalnego
Analiza języka naturalnego
 
Otwarte Miasta
Otwarte MiastaOtwarte Miasta
Otwarte Miasta
 
How to build your own google
How to build your own googleHow to build your own google
How to build your own google
 
To się w ram ie nie zmieści
To się w ram ie nie zmieściTo się w ram ie nie zmieści
To się w ram ie nie zmieści
 
Azure - Duże zbiory w chmurze
Azure - Duże zbiory w chmurzeAzure - Duże zbiory w chmurze
Azure - Duże zbiory w chmurze
 
Data Science Warsaw
Data Science WarsawData Science Warsaw
Data Science Warsaw
 
Data science w ubezpieczeniach
Data science w ubezpieczeniachData science w ubezpieczeniach
Data science w ubezpieczeniach
 
As simple as Apache Spark
As simple as Apache SparkAs simple as Apache Spark
As simple as Apache Spark
 
Big Data, Wearable, sztuczna inteligencja i ekonomia współpracy
Big  Data, Wearable, sztuczna inteligencja i ekonomia współpracyBig  Data, Wearable, sztuczna inteligencja i ekonomia współpracy
Big Data, Wearable, sztuczna inteligencja i ekonomia współpracy
 
Ask Data Anything
Ask Data AnythingAsk Data Anything
Ask Data Anything
 
Oracle Big Data Discovery - ludzka twarz Hadoop'a
Oracle Big Data Discovery - ludzka twarz Hadoop'aOracle Big Data Discovery - ludzka twarz Hadoop'a
Oracle Big Data Discovery - ludzka twarz Hadoop'a
 
Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!
Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!
Geolokalizacja i analizy przestrzenne: trzy wymiary a ile pracy dla analityka!
 
Data Exchange - the missing link in the big data value chain
Data Exchange - the missing link in the big data value chainData Exchange - the missing link in the big data value chain
Data Exchange - the missing link in the big data value chain
 
Metody logiczne w analizie danych
Metody logiczne w analizie danych Metody logiczne w analizie danych
Metody logiczne w analizie danych
 

Haven 2 0

  • 1. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HPHAVEn BigDataUseCases Mikolaj Nietz, Solution Architect Application Services Global Delivery, Hewlett-Packard
  • 2. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The changing Big Data landscape Human InformationMachine Data Business Data 10% of Information 90% of Information Annual Growth ~100% ~10%
  • 3. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Interact with and process 100% of your data seamlessly Imagine if you could… Transactional data Social media Images AudioVideoMobile Email TextsDocumentsIn-memoryHadoop Standard APIs and tools Dashboards & alerts Business intelligence Your custom appsPackaged apps Ingest Analyze Understand Machine Data Business Data Human Information Open connectors
  • 4. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Big Data Platform HAVEn HAVEn Social media IT/OT ImagesAudioVideo Transactional data Mobile Search engineEmail Texts Catalogue massive volumes of distributed data Hadoop/ HDFS Process and index all information Autonomy IDOL Analyze at extreme scale in real-time Vertica Collect & unify machine data Enterprise Security Powering HP Software + your apps nApps Documents hp.com/Haven
  • 5. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Why HAVEn? Hadoop Autonomy IDOL Vertica Enterprise Security (HP ArcSight) n – a numer of other apps „Safe Haven” = „Bezpieczna Przystań”
  • 6. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP HAVEn/Big Data Reference Architecture Rich-media data Unstructured text data Mixed-structure data Unknown-structure data Semi-structured text data Structured text data ODS EDW Data marts Hadoop HDFS Map Reduce Data integration NotOnly SQL Analytics Operational mgt. Access-in-place Meaning-based analytics (Autonomy IDOL) Autonomy value-add applications BI/ Visualization tools Analytic tools Lightweight ETL Hadoop Extended Tools Access-in-place Indexed metadata Vertica Analytics RDBMS Native analytics UDx extensions R-Functions Access-in-place Indexed metadata WWW
  • 7. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Apache Hadoop Has flexibility to store and mine any type of data • Query previously inaccessible structured and unstructured data • Not bound by single schema Excels at processing complex data • Scale-out architecture divides workloads across multiple nodes • Flexible file system eliminates ETL* bottlenecks Scales economically • Deployable on commodity hardware • Open source platform guards against vendor lock Hadoop Distributed File System (HDFS) Self-healing, high bandwidth clustered storage MapReduce Distributed Computing Framework Open source Linux-based platform for data storage and processing that is…  Scalable  Fault tolerant  Distributed Core HADOOP system components (Workloads) Like Linux, there are several distributions of Hadoop
  • 8. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Autonomy IDOL Social Media Video Audio Email Texts Mobile Transactional Data Documents XML Search Engine Images HP Autonomy IDOL Applications Autonomy Connectors eDiscovery Enterprise Search Media Monitoring Social Media Analytics Decision Support Augmented Reality Partner/ In-house apps HC Analytics Repositories Information Types Apps 500 Functions IDOL Services Multimedia Informatics Enrichment Capture InteractionAnalytics Discovery Concept Clouds Active MatchingVisualization ACA MediaBin Connected LiveVault TRIM AeD Data Protector WorkSite DigitalSafe Connectors … CloudEnterprise IDOL OS for Human Information ERP CRM Database Jive… Image HIS Data Warehouse Hadoop SharePoint
  • 9. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Seamlessly access virtually any enterprise content repository, including file systems, email, or knowledge bases 400+ connectors All data types, all content repositories – unmatched understanding HP Autonomy IDOL platform High-performance human information processing HP Autonomy IDOL Leverage the power of functions like sentiment, categorization, and clustering to deliver intelligence and insight Over 500 functions Process virtually any file type such as text (email, tweet, document), audio, video, and even people profiles & behavior 1,000+ file types Achieve big data scalability and high performance with distributable ingest and query architecture Distributable architecture
  • 10. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Vertica Real Time Analytics Platform Standard SQL Interface Native High Availability Auto Database Design Advanced Compression Column Orientation MPP Massively Parallel Processing Leverages BI, ETL, Hadoop/MapReduce and OLTP investments Automatic setup, optimization, and DB management Built-in redundancy that also speeds up queries Native DB-aware clustering on low-cost x86 Linux nodes Up to 90% space reduction using 12+ algorithms • 10x – 100x performance than classic RDBMS • High scalability from TBs to PBs • Simple integration with existing ETL and BI solutions • Superior performance on off- the-shelf hardware • Ultimate deployment flexibility • 24/7 Load and Query • Flexzone • Very close Hadoop integration • Soon-to-come: Vertica-on- Yarn
  • 11. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Why Hadoop and Vertica are complementary • Designed for Performance • Interactive Analytics • A Rich SQL Ecosystem • Designed for Fault Tolerance • Storage & Batch Processing • A Rich Programming Model Both purpose-built scalable platforms
  • 12. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Gain insight into your data in near-real time by running queries 50x-1,000x faster than legacy products Blazing fast analytics Speed, scalability, and openness at lower TCO HP Vertica Analytics platform High-performance data analytics platform purpose-built for big data HP Vertica Infinitely scale your solution by adding an unlimited number of industry-standard servers Massive scalability Protect and embrace your investment in hardware and software with built-in support for Hadoop, R, and a range of ETL and BI tools Open architecture Store 10x-30x more data per server than row databases with patented columnar compression Optimized data storage
  • 13. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Collect, normalize, and categorize machine data such as logs, events, and flows from any device, any time, anywhere from any vendor 315+ connectors Collect, store, and analyze any machine data across IT HP ArcSight Universal log management platform High-performance universal log management to consolidate machine data across IT HP ArcSight The unified machine data through filtering and parsing is enriched with rich metadata, which allows you to search machine data through simple text-based keywords without the need of domain expertise Search over 1,000,000 events per second The unified data is stored through high compression ratio in any of your existing storage formats, eliminating the need for expensive databases and DBAs Store years’ worth of data Built-in content packs, algorithms, rules, and the unified machine data help you deploy IT security, IT operations, IT GRC, and log analytics Analytics & intelligence
  • 14. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The „n”
  • 15. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Autonomy + Vertica + Tableau + HP Anywhere on Tablet
  • 16. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. German Car Manufacturer Early Warning System Business problem Detect unusual increases in the number of warranty repairs (OT warranty) as soon as they appear. Data analysis problem Detect anomalies (outliers) in time series.
  • 17. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. External Internal German Car Manufacturer Big Data Labs Warranty Repairs Landing Zone Integrated Data Analytical Record Analytical Processing Visualization HP HAVEn Platform Repairs Claims Sales Storage Parts & Production Diagnostics Reference Weather
  • 18. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Global Telecommunication Group Log Analysis Vertica ClusterNFS Hadoop Cluster Log System POC environment Vertica Hadoop Connector JDBC 3 Vertica nodes: • 2x2 core Intel XEON @ 2.7 GHz • 32 GB RAM • 9.7 TB storage Java applications Analytics & Reporting clients
  • 19. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Global Cranes Manufacturer Sensor Data Analysis Remote
  • 20. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Facebook Big Data Architecture for Log Analysis Mobile PC/Laptop Web Servers Logs Hadoop/ HDFS 2 huge Hadoop Clusters • 1.7 ExaBytes • 15000 nodes • 40000 nodes Job Scheduler Vertica Logs 15 mins Hourly Daily Legacy • 600K MR Jobs/day • 50K Informatica Jobs/day
  • 21. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Develop Operate SecureMonetize Govern HAVEn hp.com/haven Thank you!
  • 22. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Resources: • www.hp.comhaven • www.vertica.com • www.autonomy.com • www.hortonworks.com • Vertica to try: https://my.vertica.com/?redirect_to=https%3A%2F%2Fmy.vertica.com%2Fdownl oad-community-edition%2F • About HAVEn-on-demand: http://www.datacenterknowledge.com/archives/2014/12/03/hp-launches-big- data-cloud-called-haven-ondemand/