SlideShare ist ein Scribd-Unternehmen logo
1 von 30
Hadoop’s Life in Enterprise Systems Y Masatani OSS Professional Services System Platform Sector NTT DATA CORPORATION Hadoop World 2011  Nov 8 th
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Agenda
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Company Overview
Size of IT Services Market by Sectors <FY ended March 31,2011> [  Moderate Case  ]   <2010> Source: Gartner, &quot;Forecast: IT Services Japan by Industry, 1Q 2011&quot; Tsuyoshi Ebina, 20 May 2011 Note: Chart created by NTT Data based on Gartner data 42.2% 20.4% Government and healthcare Financial Enterprise, services, etc. 31.7% Other 5.7% Government and healthcare-related 15.2% 23.4% Financial Enterprise, services, etc. 61.5% Approx. 15.9% Our Shares in Markets IT Services Market in Japan NTT DATA’s Consolidated Net Sales  JPY 9.83 trillion  JPY 1.16 trillion Percent of our net sales accounted for by each customer  field /service when results are   totaled using the criteria below Government and healthcare: Central Government and Related Agencies,  Overseas Public Institutions, etc. / Local Government and Community-based Business/Healthcare Financial: Banks/Financial Unions/Insurance, Security and Credit  Corporations/Settlement Services Enterprise, services, etc.: Global IT Services Company Other: Sales not included in the above : (JPY Trillion) Approx. 6.1% Approx. 21.3%
Positioning in NTT Group NTT DATA IT solutions and Integration company USD 11 billion ,[object Object],[object Object],[object Object],* “Fortune Global 500 July 2010” (USD 1 = JPY 100) Sales Breakdown of NTT Group NTT Holdings USD 103 billion NTT EAST Regional Telephone company USD 20 billion NTT WEST Regional Telephone company USD 18 billion NTT COMMUNICATIONS Network, International Telecommunications company USD 10 billion NTT DOCOMO Mobile/Network Company USD 42 billion : Dimension Data IT communication of enterprises and service providers
[object Object],[object Object],[object Object],[object Object],Hadoop and NTT DATA
[object Object],Hadoop is Getting Hot in Japan
Popularity of Hadoop ~ 2011 Fall 3+ years  none  < 3 months  3 < 6  6 < 12 months  1 < 3 years ~50% attendees are still under research ~30% just started within 6 months
[object Object],[object Object],[object Object],[object Object],Archetype of Enterprise Hadoop
Data Processing Domains and Engines ,[object Object],Latency Size GB TB PB RDBMS Hadoop Low-Latency Serving Systems DWH, Search Engine, etc sec min hour day Big Data Processing Online Processing Enterprise Batch Processing Online Batch Processing Query & Search Processing
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Fit also to “Enterprise Batch Processing” * http://www.asakusafw.com/
Data Processing Domains and Engines ,[object Object],Latency Size GB TB PB RDBMS Hadoop Low-Latency Serving Systems DWH, Search Engine, etc sec min hour day Big Data Processing Online Processing Enterprise Batch Processing Online Batch Processing Query & Search Processing
Data Processing Domains and Engines “Revised” ,[object Object],[object Object],Big Data Processing Latency Size GB TB PB RDBMS Low-Latency Serving Systems DWH, Search Engine, etc Hadoop Enterprise Batch Processing sec min hour day Online Processing Online Batch Processing Query & Search Processing
Customers Fit into Two Areas ,[object Object],Big Data Processing Latency Size GB TB PB Enterprise Batch Processing financial media public media telcom telcom public telcom sec min hour day Online Processing Online Batch Processing Query & Search Processing
Hadoop Cluster’s Life-Cycle ,[object Object],Big Data Processing Latency Size GB TB PB Enterprise Batch Processing financial media public media telcom telcom public telcom Expansion Involvement sec min hour day Online Processing Online Batch Processing Query & Search Processing
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],“ Expansion”
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],“ Involvement” Conversion from HDFS to POSIX Conversion from HDFS to POSIX Data Processing Data Processing Hadoop
Archetype of Integration between Engines Big Data Processing Latency Size GB TB PB Enterprise Batch Processing financial media public media telcom telcom public telcom RDBMS Low-Latency Serving Systems DWH, Search Engine, etc Hadoop Raw Data Source Input Coherent Import and Export Reduction sec min hour day Online Processing Online Batch Processing Query & Search Processing
[object Object],[object Object],[object Object],[object Object],[object Object],Large Raw Data Source Input
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Coherent Import and Export with RDBMS
[object Object],[object Object],Enhanced PostgreSQL connecter for Sqoop HDFS RDBMS Map Task Map Task Map Task HDFS RDBMS Optional Map Task Map Task Map Task pg_bulkload pg_bulkoad pg_bulkload Reduce Task BEGIN INSERT INTO dest ( SELECT * FROM tmp1 ) DROP TABLE tmp1 INSERT INTO dest ( SELECT * FROM tmp2 ) DROP TABLE tmp2 INSERT INTO dest ( SELECT * FROM tmp3 ) DROP TABLE tmp3 COMMIT Issue INSERT for each chunk  of records:  INSERT INTO stg  VALUES (?, ?), (?, ?), ... INSERT INTO stg  VALUES (?, ?), (?, ?), ... ... INSERT INTO dest ( SELECT * FROM stg) CREATE TABLE   tmp3(LIKE dest INCLUDING CONSTRAINTS) Sqoop (baseline implementation) Specialized implementation for PostgreSQL Exclude error records into a separate file Staging Table File Split File Split File Split Destination Table File Split File Split File Split Destination Table tmp1 tmp2 tmp3
Feature of Sqoop PostgreSQL Connector 1 Robust and efficient direct export using “pg_bulkload” ,[object Object],[object Object],[object Object],2 Tune export using PostgreSQL COPY  ,[object Object],[object Object],3 Import using “ctid” for string type key value ,[object Object],5 Balanced import using statistical information ,[object Object],4 Tune deletion method for staging table ,[object Object]
[object Object],[object Object],[object Object],[object Object],Integration with Low-Latency Serving System  * http://www.ntt.co.jp/RD/OFIS/index_en.html
[object Object],[object Object],Prototype of Hadoop and GPGPU Integration Data Collection Feature Data Extraction Kmeans Clustering Result of Clustering Feature Data Compression Compressed Feature Data: - ROWs: 1000~100000 - COLs: 100~1000 - SIZE: order of ~GB Feature Data: - ROWs: 1000~100000 - COLs: 10000~100000 - SIZE:  order of ~10GB Input Data (Query Log): - Unique User: 30,000[UU/H] - SIZE:  order of ~TB Hadoop Slave Hadoop Master Flume Collector Flume Master /ZooKeeper GPU Server Raw Data Source
Breakdown of Elapsed Time for K-means 24 cores 3 nodes  256 cores  1 node
[object Object],[object Object],Connector Integration Beyond ,[object Object],[object Object],[object Object],[object Object],[object Object],Common development in  and  Hadoop Cluster for Enterprise Batch Processing Backup and Recovery POSIX HDFS APIs System B System A
Copyright 2011 FUJITSU LIMITED Enhanced Storage Architecture Established storage management technology (memory caching and disk I/O scheduling) and enhanced dedicated network enables boosted HDFS performance Local FS Mem CPU Extract Disk I/O bandwidth as of Locality Local FS Mem CPU Local FS Mem CPU Mem CPU Mem CPU Mem CPU Meshed network (40Gb b/w) Pros: Achieve  Read 5x and Write 10x  performance based on a financial enterprise batch benchmark case compared to local disk HDFS. Cons: Limited scalability (up to 40~50 nodes based on the prototype configuration, will be extended to ~120) Enhanced Bandwidth between Nodes and Storage Storage File system supports HDFS APIs
[object Object],Connector Integration Beyond ,[object Object],[object Object],Common development in  and  Can Eliminate This Overhead Conversion from HDFS to POSIX Hadoop
Hadoop with Enterprise Market ,[object Object],[object Object]
Thank you contact: hadoop at kits.nttdata.co.jp

Weitere ähnliche Inhalte

Was ist angesagt?

Best Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop Professionals
Best Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop ProfessionalsBest Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop Professionals
Best Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop ProfessionalsCloudera, Inc.
 
ETL big data with apache hadoop
ETL big data with apache hadoopETL big data with apache hadoop
ETL big data with apache hadoopMaulik Thaker
 
Big Data Warehousing: Pig vs. Hive Comparison
Big Data Warehousing: Pig vs. Hive ComparisonBig Data Warehousing: Pig vs. Hive Comparison
Big Data Warehousing: Pig vs. Hive ComparisonCaserta
 
Distributed Data Analysis with Hadoop and R - Strangeloop 2011
Distributed Data Analysis with Hadoop and R - Strangeloop 2011Distributed Data Analysis with Hadoop and R - Strangeloop 2011
Distributed Data Analysis with Hadoop and R - Strangeloop 2011Jonathan Seidman
 
Big data Hadoop Analytic and Data warehouse comparison guide
Big data Hadoop Analytic and Data warehouse comparison guideBig data Hadoop Analytic and Data warehouse comparison guide
Big data Hadoop Analytic and Data warehouse comparison guideDanairat Thanabodithammachari
 
Hadoop Powers Modern Enterprise Data Architectures
Hadoop Powers Modern Enterprise Data ArchitecturesHadoop Powers Modern Enterprise Data Architectures
Hadoop Powers Modern Enterprise Data ArchitecturesDataWorks Summit
 
2014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part12014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part1Adam Muise
 
Distributed Data Analysis with Hadoop and R - OSCON 2011
Distributed Data Analysis with Hadoop and R - OSCON 2011Distributed Data Analysis with Hadoop and R - OSCON 2011
Distributed Data Analysis with Hadoop and R - OSCON 2011Jonathan Seidman
 
Big data processing with apache spark part1
Big data processing with apache spark   part1Big data processing with apache spark   part1
Big data processing with apache spark part1Abbas Maazallahi
 
Integrated Data Warehouse with Hadoop and Oracle Database
Integrated Data Warehouse with Hadoop and Oracle DatabaseIntegrated Data Warehouse with Hadoop and Oracle Database
Integrated Data Warehouse with Hadoop and Oracle DatabaseGwen (Chen) Shapira
 
Building a Big Data platform with the Hadoop ecosystem
Building a Big Data platform with the Hadoop ecosystemBuilding a Big Data platform with the Hadoop ecosystem
Building a Big Data platform with the Hadoop ecosystemGregg Barrett
 
Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which DataWorks Summit
 
Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...
Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...
Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...Mahantesh Angadi
 

Was ist angesagt? (20)

Best Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop Professionals
Best Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop ProfessionalsBest Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop Professionals
Best Practices for the Hadoop Data Warehouse: EDW 101 for Hadoop Professionals
 
ETL big data with apache hadoop
ETL big data with apache hadoopETL big data with apache hadoop
ETL big data with apache hadoop
 
Big Data Warehousing: Pig vs. Hive Comparison
Big Data Warehousing: Pig vs. Hive ComparisonBig Data Warehousing: Pig vs. Hive Comparison
Big Data Warehousing: Pig vs. Hive Comparison
 
A data analyst view of Bigdata
A data analyst view of Bigdata A data analyst view of Bigdata
A data analyst view of Bigdata
 
Distributed Data Analysis with Hadoop and R - Strangeloop 2011
Distributed Data Analysis with Hadoop and R - Strangeloop 2011Distributed Data Analysis with Hadoop and R - Strangeloop 2011
Distributed Data Analysis with Hadoop and R - Strangeloop 2011
 
Big data Hadoop Analytic and Data warehouse comparison guide
Big data Hadoop Analytic and Data warehouse comparison guideBig data Hadoop Analytic and Data warehouse comparison guide
Big data Hadoop Analytic and Data warehouse comparison guide
 
Hadoop Powers Modern Enterprise Data Architectures
Hadoop Powers Modern Enterprise Data ArchitecturesHadoop Powers Modern Enterprise Data Architectures
Hadoop Powers Modern Enterprise Data Architectures
 
Big Data Concepts
Big Data ConceptsBig Data Concepts
Big Data Concepts
 
Big data ppt
Big data pptBig data ppt
Big data ppt
 
Understanding hdfs
Understanding hdfsUnderstanding hdfs
Understanding hdfs
 
Big data
Big dataBig data
Big data
 
2014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part12014 sept 26_thug_lambda_part1
2014 sept 26_thug_lambda_part1
 
Distributed Data Analysis with Hadoop and R - OSCON 2011
Distributed Data Analysis with Hadoop and R - OSCON 2011Distributed Data Analysis with Hadoop and R - OSCON 2011
Distributed Data Analysis with Hadoop and R - OSCON 2011
 
Big data processing with apache spark part1
Big data processing with apache spark   part1Big data processing with apache spark   part1
Big data processing with apache spark part1
 
Integrated Data Warehouse with Hadoop and Oracle Database
Integrated Data Warehouse with Hadoop and Oracle DatabaseIntegrated Data Warehouse with Hadoop and Oracle Database
Integrated Data Warehouse with Hadoop and Oracle Database
 
Big data hadoop rdbms
Big data hadoop rdbmsBig data hadoop rdbms
Big data hadoop rdbms
 
Building a Big Data platform with the Hadoop ecosystem
Building a Big Data platform with the Hadoop ecosystemBuilding a Big Data platform with the Hadoop ecosystem
Building a Big Data platform with the Hadoop ecosystem
 
Big data concepts
Big data conceptsBig data concepts
Big data concepts
 
Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which Hadoop and the Data Warehouse: When to Use Which
Hadoop and the Data Warehouse: When to Use Which
 
Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...
Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...
Introduction and Overview of BigData, Hadoop, Distributed Computing - BigData...
 

Andere mochten auch

NTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポート
NTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポートNTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポート
NTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポートNTT DATA OSS Professional Services
 
分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み
分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み
分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組みNTT DATA OSS Professional Services
 
Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)
Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)
Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)NTT DATA OSS Professional Services
 
Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)
Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)
Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)NTT DATA OSS Professional Services
 
Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)
Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)
Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)NTT DATA OSS Professional Services
 
Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)
Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)
Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)NTT DATA OSS Professional Services
 
データ活用をもっともっと円滑に! ~データ処理・分析基盤編を少しだけ~
データ活用をもっともっと円滑に!~データ処理・分析基盤編を少しだけ~データ活用をもっともっと円滑に!~データ処理・分析基盤編を少しだけ~
データ活用をもっともっと円滑に! ~データ処理・分析基盤編を少しだけ~NTT DATA OSS Professional Services
 

Andere mochten auch (14)

NTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポート
NTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポートNTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポート
NTTデータにおけるHadoopへの取り組み & Hadoop Summit 2010 レポート
 
分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み
分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み
分散処理基盤Apache Hadoopの現状と、NTTデータのHadoopに対する取り組み
 
Hadoop Conference Japan 2009 - NTT Data
Hadoop Conference Japan 2009 - NTT DataHadoop Conference Japan 2009 - NTT Data
Hadoop Conference Japan 2009 - NTT Data
 
Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)
Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)
Hadoop 2.6の最新機能(Cloudera World Tokyo 2014 LT講演資料)
 
HTrace: Tracing in HBase and HDFS (HBase Meetup)
HTrace: Tracing in HBase and HDFS (HBase Meetup)HTrace: Tracing in HBase and HDFS (HBase Meetup)
HTrace: Tracing in HBase and HDFS (HBase Meetup)
 
Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)
Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)
Sparkコミュニティに飛び込もう!(Spark Meetup Tokyo 2015 講演資料、NTTデータ 猿田 浩輔)
 
Hadoop2.6の最新機能+
Hadoop2.6の最新機能+Hadoop2.6の最新機能+
Hadoop2.6の最新機能+
 
Hadoop ecosystem NTTDATA osc15tk
Hadoop ecosystem NTTDATA osc15tkHadoop ecosystem NTTDATA osc15tk
Hadoop ecosystem NTTDATA osc15tk
 
Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)
Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)
Ansibleで構成管理始める人のモチベーションをあげたい! (Cloudera World Tokyo 2014LT講演資料)
 
Apache Spark 1000 nodes NTT DATA
Apache Spark 1000 nodes NTT DATAApache Spark 1000 nodes NTT DATA
Apache Spark 1000 nodes NTT DATA
 
Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)
Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)
Sparkをノートブックにまとめちゃおう。Zeppelinでね!(Hadoopソースコードリーディング 第19回 発表資料)
 
Apache Hadoop 2.8.0 の新機能 (抜粋)
Apache Hadoop 2.8.0 の新機能 (抜粋)Apache Hadoop 2.8.0 の新機能 (抜粋)
Apache Hadoop 2.8.0 の新機能 (抜粋)
 
データ活用をもっともっと円滑に! ~データ処理・分析基盤編を少しだけ~
データ活用をもっともっと円滑に!~データ処理・分析基盤編を少しだけ~データ活用をもっともっと円滑に!~データ処理・分析基盤編を少しだけ~
データ活用をもっともっと円滑に! ~データ処理・分析基盤編を少しだけ~
 
Spark MLlibではじめるスケーラブルな機械学習
Spark MLlibではじめるスケーラブルな機械学習Spark MLlibではじめるスケーラブルな機械学習
Spark MLlibではじめるスケーラブルな機械学習
 

Ähnlich wie Hadoop World 2011: Hadoop’s Life in Enterprise Systems - Y Masatani, NTTData

Big Data Taiwan 2014 Track2-2: Informatica Big Data Solution
Big Data Taiwan 2014 Track2-2: Informatica Big Data SolutionBig Data Taiwan 2014 Track2-2: Informatica Big Data Solution
Big Data Taiwan 2014 Track2-2: Informatica Big Data SolutionEtu Solution
 
Deutsche Telekom on Big Data
Deutsche Telekom on Big DataDeutsche Telekom on Big Data
Deutsche Telekom on Big DataDataWorks Summit
 
Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010nzhang
 
Big data Hadoop presentation
Big data  Hadoop  presentation Big data  Hadoop  presentation
Big data Hadoop presentation Shivanee garg
 
Stratebi Big Data
Stratebi Big DataStratebi Big Data
Stratebi Big DataStratebi
 
The Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionThe Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionMongoDB
 
La creación de una capa operacional con MongoDB
La creación de una capa operacional con MongoDBLa creación de una capa operacional con MongoDB
La creación de una capa operacional con MongoDBMongoDB
 
Finding the needles in the haystack. An Overview of Analyzing Big Data with H...
Finding the needles in the haystack. An Overview of Analyzing Big Data with H...Finding the needles in the haystack. An Overview of Analyzing Big Data with H...
Finding the needles in the haystack. An Overview of Analyzing Big Data with H...Chris Baglieri
 
Big Data
Big DataBig Data
Big DataNGDATA
 
IEEE International Conference on Data Engineering 2015
IEEE International Conference on Data Engineering 2015IEEE International Conference on Data Engineering 2015
IEEE International Conference on Data Engineering 2015Yousun Jeong
 
Simplifying Big Data ETL with Talend
Simplifying Big Data ETL with TalendSimplifying Big Data ETL with Talend
Simplifying Big Data ETL with TalendEdureka!
 
Mammothdb - Public VC Pitchdeck!
Mammothdb - Public VC Pitchdeck!Mammothdb - Public VC Pitchdeck!
Mammothdb - Public VC Pitchdeck!Steve Keil
 
Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaJeffrey T. Pollock
 
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationMaximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationDenodo
 
Big data data lake and beyond
Big data data lake and beyond Big data data lake and beyond
Big data data lake and beyond Rajesh Kumar
 
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...Cloudera, Inc.
 

Ähnlich wie Hadoop World 2011: Hadoop’s Life in Enterprise Systems - Y Masatani, NTTData (20)

Big Data Taiwan 2014 Track2-2: Informatica Big Data Solution
Big Data Taiwan 2014 Track2-2: Informatica Big Data SolutionBig Data Taiwan 2014 Track2-2: Informatica Big Data Solution
Big Data Taiwan 2014 Track2-2: Informatica Big Data Solution
 
Deutsche Telekom on Big Data
Deutsche Telekom on Big DataDeutsche Telekom on Big Data
Deutsche Telekom on Big Data
 
Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010Hive @ Hadoop day seattle_2010
Hive @ Hadoop day seattle_2010
 
Big data Hadoop presentation
Big data  Hadoop  presentation Big data  Hadoop  presentation
Big data Hadoop presentation
 
Stratebi Big Data
Stratebi Big DataStratebi Big Data
Stratebi Big Data
 
The Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reductionThe Double win business transformation and in-year ROI and TCO reduction
The Double win business transformation and in-year ROI and TCO reduction
 
La creación de una capa operacional con MongoDB
La creación de una capa operacional con MongoDBLa creación de una capa operacional con MongoDB
La creación de una capa operacional con MongoDB
 
Finding the needles in the haystack. An Overview of Analyzing Big Data with H...
Finding the needles in the haystack. An Overview of Analyzing Big Data with H...Finding the needles in the haystack. An Overview of Analyzing Big Data with H...
Finding the needles in the haystack. An Overview of Analyzing Big Data with H...
 
Big Data
Big DataBig Data
Big Data
 
IEEE International Conference on Data Engineering 2015
IEEE International Conference on Data Engineering 2015IEEE International Conference on Data Engineering 2015
IEEE International Conference on Data Engineering 2015
 
Simplifying Big Data ETL with Talend
Simplifying Big Data ETL with TalendSimplifying Big Data ETL with Talend
Simplifying Big Data ETL with Talend
 
Mammothdb - Public VC Pitchdeck!
Mammothdb - Public VC Pitchdeck!Mammothdb - Public VC Pitchdeck!
Mammothdb - Public VC Pitchdeck!
 
The CDO Agenda: how data architecture can help?
The CDO Agenda: how data architecture can help?The CDO Agenda: how data architecture can help?
The CDO Agenda: how data architecture can help?
 
Tera stream ETL
Tera stream ETLTera stream ETL
Tera stream ETL
 
Soma_Chakraborty (1)
Soma_Chakraborty (1)Soma_Chakraborty (1)
Soma_Chakraborty (1)
 
Big data analysis concepts and references
Big data analysis concepts and referencesBig data analysis concepts and references
Big data analysis concepts and references
 
Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
 
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationMaximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
 
Big data data lake and beyond
Big data data lake and beyond Big data data lake and beyond
Big data data lake and beyond
 
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
 

Mehr von Cloudera, Inc.

Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptxPartner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptxCloudera, Inc.
 
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists Cloudera, Inc.
 
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards FinalistsCloudera, Inc.
 
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019Cloudera, Inc.
 
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19Cloudera, Inc.
 
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19Cloudera, Inc.
 
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19Cloudera, Inc.
 
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19Cloudera, Inc.
 
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Cloudera, Inc.
 
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19Cloudera, Inc.
 
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Cloudera, Inc.
 
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18Cloudera, Inc.
 
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3Cloudera, Inc.
 
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2Cloudera, Inc.
 
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1Cloudera, Inc.
 
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the PlatformExtending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the PlatformCloudera, Inc.
 
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18Cloudera, Inc.
 
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360Cloudera, Inc.
 
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18Cloudera, Inc.
 
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18Cloudera, Inc.
 

Mehr von Cloudera, Inc. (20)

Partner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptxPartner Briefing_January 25 (FINAL).pptx
Partner Briefing_January 25 (FINAL).pptx
 
Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists Cloudera Data Impact Awards 2021 - Finalists
Cloudera Data Impact Awards 2021 - Finalists
 
2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists2020 Cloudera Data Impact Awards Finalists
2020 Cloudera Data Impact Awards Finalists
 
Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019Edc event vienna presentation 1 oct 2019
Edc event vienna presentation 1 oct 2019
 
Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19Machine Learning with Limited Labeled Data 4/3/19
Machine Learning with Limited Labeled Data 4/3/19
 
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19Data Driven With the Cloudera Modern Data Warehouse 3.19.19
Data Driven With the Cloudera Modern Data Warehouse 3.19.19
 
Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19Introducing Cloudera DataFlow (CDF) 2.13.19
Introducing Cloudera DataFlow (CDF) 2.13.19
 
Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19Introducing Cloudera Data Science Workbench for HDP 2.12.19
Introducing Cloudera Data Science Workbench for HDP 2.12.19
 
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
Shortening the Sales Cycle with a Modern Data Warehouse 1.30.19
 
Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19Leveraging the cloud for analytics and machine learning 1.29.19
Leveraging the cloud for analytics and machine learning 1.29.19
 
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
Modernizing the Legacy Data Warehouse – What, Why, and How 1.23.19
 
Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18Leveraging the Cloud for Big Data Analytics 12.11.18
Leveraging the Cloud for Big Data Analytics 12.11.18
 
Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3Modern Data Warehouse Fundamentals Part 3
Modern Data Warehouse Fundamentals Part 3
 
Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2Modern Data Warehouse Fundamentals Part 2
Modern Data Warehouse Fundamentals Part 2
 
Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1Modern Data Warehouse Fundamentals Part 1
Modern Data Warehouse Fundamentals Part 1
 
Extending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the PlatformExtending Cloudera SDX beyond the Platform
Extending Cloudera SDX beyond the Platform
 
Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18Federated Learning: ML with Privacy on the Edge 11.15.18
Federated Learning: ML with Privacy on the Edge 11.15.18
 
Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360Analyst Webinar: Doing a 180 on Customer 360
Analyst Webinar: Doing a 180 on Customer 360
 
Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18Build a modern platform for anti-money laundering 9.19.18
Build a modern platform for anti-money laundering 9.19.18
 
Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18Introducing the data science sandbox as a service 8.30.18
Introducing the data science sandbox as a service 8.30.18
 

Kürzlich hochgeladen

Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03DallasHaselhorst
 
Marketplace and Quality Assurance Presentation - Vincent Chirchir
Marketplace and Quality Assurance Presentation - Vincent ChirchirMarketplace and Quality Assurance Presentation - Vincent Chirchir
Marketplace and Quality Assurance Presentation - Vincent Chirchirictsugar
 
8447779800, Low rate Call girls in Tughlakabad Delhi NCR
8447779800, Low rate Call girls in Tughlakabad Delhi NCR8447779800, Low rate Call girls in Tughlakabad Delhi NCR
8447779800, Low rate Call girls in Tughlakabad Delhi NCRashishs7044
 
Youth Involvement in an Innovative Coconut Value Chain by Mwalimu Menza
Youth Involvement in an Innovative Coconut Value Chain by Mwalimu MenzaYouth Involvement in an Innovative Coconut Value Chain by Mwalimu Menza
Youth Involvement in an Innovative Coconut Value Chain by Mwalimu Menzaictsugar
 
Organizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessOrganizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessSeta Wicaksana
 
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607dollysharma2066
 
Intro to BCG's Carbon Emissions Benchmark_vF.pdf
Intro to BCG's Carbon Emissions Benchmark_vF.pdfIntro to BCG's Carbon Emissions Benchmark_vF.pdf
Intro to BCG's Carbon Emissions Benchmark_vF.pdfpollardmorgan
 
/:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In...
/:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In.../:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In...
/:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In...lizamodels9
 
Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...
Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...
Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...lizamodels9
 
Market Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 EditionMarket Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 EditionMintel Group
 
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,noida100girls
 
Ten Organizational Design Models to align structure and operations to busines...
Ten Organizational Design Models to align structure and operations to busines...Ten Organizational Design Models to align structure and operations to busines...
Ten Organizational Design Models to align structure and operations to busines...Seta Wicaksana
 
Buy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail AccountsBuy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail AccountsBuy Verified Accounts
 
8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCR8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCRashishs7044
 
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCRashishs7044
 
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCRashishs7044
 
2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis Usage2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis UsageNeil Kimberley
 
FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607dollysharma2066
 
APRIL2024_UKRAINE_xml_0000000000000 .pdf
APRIL2024_UKRAINE_xml_0000000000000 .pdfAPRIL2024_UKRAINE_xml_0000000000000 .pdf
APRIL2024_UKRAINE_xml_0000000000000 .pdfRbc Rbcua
 

Kürzlich hochgeladen (20)

Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03Cybersecurity Awareness Training Presentation v2024.03
Cybersecurity Awareness Training Presentation v2024.03
 
Japan IT Week 2024 Brochure by 47Billion (English)
Japan IT Week 2024 Brochure by 47Billion (English)Japan IT Week 2024 Brochure by 47Billion (English)
Japan IT Week 2024 Brochure by 47Billion (English)
 
Marketplace and Quality Assurance Presentation - Vincent Chirchir
Marketplace and Quality Assurance Presentation - Vincent ChirchirMarketplace and Quality Assurance Presentation - Vincent Chirchir
Marketplace and Quality Assurance Presentation - Vincent Chirchir
 
8447779800, Low rate Call girls in Tughlakabad Delhi NCR
8447779800, Low rate Call girls in Tughlakabad Delhi NCR8447779800, Low rate Call girls in Tughlakabad Delhi NCR
8447779800, Low rate Call girls in Tughlakabad Delhi NCR
 
Youth Involvement in an Innovative Coconut Value Chain by Mwalimu Menza
Youth Involvement in an Innovative Coconut Value Chain by Mwalimu MenzaYouth Involvement in an Innovative Coconut Value Chain by Mwalimu Menza
Youth Involvement in an Innovative Coconut Value Chain by Mwalimu Menza
 
Organizational Structure Running A Successful Business
Organizational Structure Running A Successful BusinessOrganizational Structure Running A Successful Business
Organizational Structure Running A Successful Business
 
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
(Best) ENJOY Call Girls in Faridabad Ex | 8377087607
 
Intro to BCG's Carbon Emissions Benchmark_vF.pdf
Intro to BCG's Carbon Emissions Benchmark_vF.pdfIntro to BCG's Carbon Emissions Benchmark_vF.pdf
Intro to BCG's Carbon Emissions Benchmark_vF.pdf
 
/:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In...
/:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In.../:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In...
/:Call Girls In Indirapuram Ghaziabad ➥9990211544 Independent Best Escorts In...
 
Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...
Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...
Call Girls In Radisson Blu Hotel New Delhi Paschim Vihar ❤️8860477959 Escorts...
 
Market Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 EditionMarket Sizes Sample Report - 2024 Edition
Market Sizes Sample Report - 2024 Edition
 
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
BEST Call Girls In Greater Noida ✨ 9773824855 ✨ Escorts Service In Delhi Ncr,
 
Ten Organizational Design Models to align structure and operations to busines...
Ten Organizational Design Models to align structure and operations to busines...Ten Organizational Design Models to align structure and operations to busines...
Ten Organizational Design Models to align structure and operations to busines...
 
Buy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail AccountsBuy gmail accounts.pdf Buy Old Gmail Accounts
Buy gmail accounts.pdf Buy Old Gmail Accounts
 
8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCR8447779800, Low rate Call girls in Saket Delhi NCR
8447779800, Low rate Call girls in Saket Delhi NCR
 
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
8447779800, Low rate Call girls in New Ashok Nagar Delhi NCR
 
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
8447779800, Low rate Call girls in Shivaji Enclave Delhi NCR
 
2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis Usage2024 Numerator Consumer Study of Cannabis Usage
2024 Numerator Consumer Study of Cannabis Usage
 
FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607FULL ENJOY Call girls in Paharganj Delhi | 8377087607
FULL ENJOY Call girls in Paharganj Delhi | 8377087607
 
APRIL2024_UKRAINE_xml_0000000000000 .pdf
APRIL2024_UKRAINE_xml_0000000000000 .pdfAPRIL2024_UKRAINE_xml_0000000000000 .pdf
APRIL2024_UKRAINE_xml_0000000000000 .pdf
 

Hadoop World 2011: Hadoop’s Life in Enterprise Systems - Y Masatani, NTTData

  • 1. Hadoop’s Life in Enterprise Systems Y Masatani OSS Professional Services System Platform Sector NTT DATA CORPORATION Hadoop World 2011 Nov 8 th
  • 2.
  • 3.
  • 4. Size of IT Services Market by Sectors <FY ended March 31,2011> [ Moderate Case ] <2010> Source: Gartner, &quot;Forecast: IT Services Japan by Industry, 1Q 2011&quot; Tsuyoshi Ebina, 20 May 2011 Note: Chart created by NTT Data based on Gartner data 42.2% 20.4% Government and healthcare Financial Enterprise, services, etc. 31.7% Other 5.7% Government and healthcare-related 15.2% 23.4% Financial Enterprise, services, etc. 61.5% Approx. 15.9% Our Shares in Markets IT Services Market in Japan NTT DATA’s Consolidated Net Sales JPY 9.83 trillion JPY 1.16 trillion Percent of our net sales accounted for by each customer field /service when results are totaled using the criteria below Government and healthcare: Central Government and Related Agencies, Overseas Public Institutions, etc. / Local Government and Community-based Business/Healthcare Financial: Banks/Financial Unions/Insurance, Security and Credit Corporations/Settlement Services Enterprise, services, etc.: Global IT Services Company Other: Sales not included in the above : (JPY Trillion) Approx. 6.1% Approx. 21.3%
  • 5.
  • 6.
  • 7.
  • 8. Popularity of Hadoop ~ 2011 Fall 3+ years none < 3 months 3 < 6 6 < 12 months 1 < 3 years ~50% attendees are still under research ~30% just started within 6 months
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18. Archetype of Integration between Engines Big Data Processing Latency Size GB TB PB Enterprise Batch Processing financial media public media telcom telcom public telcom RDBMS Low-Latency Serving Systems DWH, Search Engine, etc Hadoop Raw Data Source Input Coherent Import and Export Reduction sec min hour day Online Processing Online Batch Processing Query & Search Processing
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25. Breakdown of Elapsed Time for K-means 24 cores 3 nodes 256 cores 1 node
  • 26.
  • 27. Copyright 2011 FUJITSU LIMITED Enhanced Storage Architecture Established storage management technology (memory caching and disk I/O scheduling) and enhanced dedicated network enables boosted HDFS performance Local FS Mem CPU Extract Disk I/O bandwidth as of Locality Local FS Mem CPU Local FS Mem CPU Mem CPU Mem CPU Mem CPU Meshed network (40Gb b/w) Pros: Achieve Read 5x and Write 10x performance based on a financial enterprise batch benchmark case compared to local disk HDFS. Cons: Limited scalability (up to 40~50 nodes based on the prototype configuration, will be extended to ~120) Enhanced Bandwidth between Nodes and Storage Storage File system supports HDFS APIs
  • 28.
  • 29.
  • 30. Thank you contact: hadoop at kits.nttdata.co.jp

Hinweis der Redaktion

  1. Hadoop’s Life in Enterprise Systems NTT DATA has been providing Hadoop professional services for enterprise customers for years. In this talk we will categorize Hadoop integration cases based on our experience and illustrate archetypal design practices how Hadoop clusters are deployed into existing infrastructure and services. We will also present enhancement cases motivated by customer’s demand including GPU for big math, HDFS capable storage system, etc. Y Masatani Senior Specialist NTT DATA Masatani is a senior specialist at System Platforms Sector in NTT DATA Corporation. He has more than 15 years experience in software engineering and Internet services. He has been directed OSS professional services unit from 2006 and delivering technical services and developing platform solutions. The team first became acquainted with Hadoop late 2007 and started operational support services from mid 2008.
  2. Who we are? The situation of Hadoop in Japan Our experience .. What have been learnt , What have we observe in our customers and their clusters. More than fingers of both arms and both legs.
  3. We will introduce who we are? 11.6 B, SI, Consulting , Outsourcing
  4. Left Middle Right .. All rounder in Japanese IT Service Market
  5. Nov 2009 – there was one session from Cloudera 2 nd one takes 15months 3 rd one comes earlier in 7 months there were Cloudera, Horton, MapR from US. Hope will have Hadoop World Japan or ASIA in the near future..
  6. Regarding the popularity and deployment of Hadoop. It is not wide and not matured enough yet but APPARENTLY it is accelerated in this year
  7. Let’s look at landscape first…
  8. Let’s look at “data processing domains” and “applicable engines” データの流れ・変化と処理内容の変遷 Data warehouse servers Mid-tire servers
  9. Let’s talk about our experience Parallel processing based on “data locality” That would be beneficial large amount of data and also repetitive sweeping of data. Receipt processing on healthcare / insurance
  10. So, the landscape changes from here to here..
  11. データの流れ・変化と処理内容の変遷 According to our customer’s cases.. Data warehouse servers Mid-tire servers
  12. データの流れ・変化と処理内容の変遷 According to our customer’s cases.. Data warehouse servers Mid-tire servers
  13. We have been over 3 years support for customers. Then the oldest clusters are going to be renewed and expanded We called these area as “Frontiers” and “Establishment” last year.. We call these as “Involvement” and “Expansion” after some more reasoning… Here is the story
  14. These groups are not different in processing domain, but also in Life-Cycle We haven’t seen huge cases yet.. Let’s talk about our experience Parallel processing based on “data locality” That would be beneficial large amount of data and also repetitive sweeping of data. Receipt processing on healthcare / insurance
  15. Many clusters or a Big cluster Hadoop cluster itself has good scalability and expandability..
  16. Do we have flexible / useful scalability ??? According to our customer’s cases.. Data warehouse servers Mid-tire servers
  17. Parallel processing based on “data locality” That would be beneficial large amount of data and also repetitive sweeping of data. Receipt processing on healthcare / insurance
  18. Parallel processing based on “data locality” That would be beneficial large amount of data and also repetitive sweeping of data. Receipt processing on healthcare / insurance PostgreSQL is more polu
  19. 利点 高速 DB サーバの負荷が小さい。 (WAL 、共有バッファをバイパスできる。 ) エラーが発生するレコードを飛ばしてデータを RDB にロードできる。 エラーが発生するレコードがどれかをログから確認できる。 エラーが発生しても export 先テーブルにゴミが残らない。 欠点 DB の管理者権限がなければ使いにくい 各 Map タスクで一時テーブルの作成、削除を行う。 pg_bulkoad のログは DB サーバ側に出力される。 全スレーブノードに pg_bulkload をインストールする必要がある。
  20. RDBMS serves online-batch processing
  21. Copyright 2011 FUJITSU LIMITED