Talend Big Data Capabilities Overview

1. Talend: Solutions Overview
Presenter: Rajan Kanitkar

2. Talend Big Data Overview

3. © Talend 2012
The Drivers for Big Data
Volume
Velocity
Variety

4. © Talend 2012
The defacto standard for big data processing
How to process big data?

5. © Talend 2012
Apache Hadoop, an open-source software library, is a framework
that allows for the distributed processing of large data sets across
clusters of commodity hardware using simple programming
models. It is designed to scale up from single servers to
thousands of machines, each offering local computation and
storage.
What is Hadoop?

6. © Talend 2012
The Big Data Ecosystem
Hadoop: the core project
HDFS: the Hadoop Distributed File System
MapReduce: the software framework for distributed processing
of large data sets
Hive: a data warehouse infrastructure that provides data
summarization and a querying language
Pig: a high-level data-flow language and execution framework
for parallel computation
HBase: this is the Hadoop database. Use it when you need
random, realtime read/write access to your Big Data
And many many more: Sqoop, HCatalog, Zookeeper, Oozie,
Cassandra, MongoDB, Flume, Impala, Stinger, Neo4J, etc.
Thanks to you all!
Google, Amazon, Facebook, Twitter, Yahoo, 10gen, Cloudera,
Hortonworks, MapR, etc.

7. Talend Big Data Overview

8. © Talend 2012
Key differentiator Of Our Next Gen Architecture…
JAVA
ETL
Day-to-day
integration
Run everywhere
SQL
ELT
DW
appliance
Teradata, Netezza…
MapReduce
Hadoop
Highly
Scalable
Hadoop Grid
CAMEL
CAMEL
Message
transform-
ation
High Frequency
 No black-box engine
 Enables light-weight
distributed, customizable and
parallelizable run time
 Standards-Based
Code Generator

9. © Talend 2012 12
Talend Unique Integration Solution
Consolidated
metadata & project
information
Repository
2
Web-based
deployment &
scheduling
Deployment
3 Same container for
batch processing,
message routing &
services
Execution
4
Single web-based
monitoring console
Monitoring
5
Comprehensive
Eclipse-based
user interface
1
Studio
Data
Quality
Data
Integration MDM ESB BPM
Best-of-
Breed
Solutions
+
Talend
Unified
Platform
=
Unique
Integration
Solution

10. © Talend 2012
Talend Big Data Product Strategy
Big Data Integration
▶ Land data in a Big Data cluster without coding
▶ Code generation for MapReduce, HDFS, Hbase, Pig, Hive, Hcatalog, etc.
Big Data Manipulation
▶ Simplify manipulation, such as sort and filter
▶ Computational expensive functions using Hadoop
Big Data Quality & Governance
▶ Identify linkages & duplicates, validate big data
▶ Match component, execute basic quality features
Big Data Project Management
▶ Place frameworks around big data projects
▶ Common Repository, scheduling, monitoring
strategic pillars

11. © Talend 2012
…an open source
ecosystem
Talend Open Studio for Big Data
• Improves efficiency of big data job design
with graphic interface
• Generates Hadoop code and run transforms
inside Hadoop
• Native support for HDFS, Pig, Hbase,
Hcatalog, Sqoop and Hive
• 100% open source under an Apache License
• Standards based
Pig
Vision: Democratize big data

12. © Talend 2012
…an open source
ecosystem
Talend Platform for Big Data
• Builds on Talend Open Studio for Big Data
• Adds data quality, advanced scalability and
management functions
• MapReduce massively parallel data
processing
• Shared Repository and remote
deployment
• Data quality and profiling
• Data cleansing
• Reporting and dashboards
• Commercial support, warranty/IP
indemnity under a subscription license
Pig
Vision: Democratize big data

13. © Talend 2012
Talend Big Data Partnerships
Hadoop DistributionsTalend Big Data
Partners

14. © Talend 2012
Demonstration: ETL for Big Data with Talend
Extract
Transform
Load

15. Talend Demo
2013