Zeppelin Interpreters
PSQL (to became JDBC in 0.6.x)
Geode
SpringXD
Apache Ambari
Zeppelin Service
Geode, HAWQ and Spring XD services
Webpage Embedder View
5. Technical Stack
Apache HDFS Data Lake - PHD or HDP Hadoop
Apache HAWQ SQL on Hadoop (OLAP)
Apache Geode In-memory data grid (OLTP)
Spring XD Integration and Streaming Runtime
Apache Ambari Manages All Clusters
Apache Zeppelin Web UI for interaction with Data Systems
Hadoop/HDFS
Geode HAWQ
SpringXD
Ambari
Zeppelin
7. Apache Geode
⢠Cache - Performance / Consistency / Resiliency
⢠Region - Highly available, redundant, distributed
Map
China Railway
Corporation
5,700 train stations
4.5 million tickets per day
20 million daily users
1.4 billion page views per day
40,000 visits per second
Indian Railways
7,000 stations
72,000 miles of track
23 million passengers daily
120,000 concurrent users
10,000 transactions per minute
8. Apache HAWQ
⢠Built around a Greenplum MPP DB
⢠100% ANSI SQL compliant: SQL-92/99/2003âŚ
⢠ODBC and JDBC
⢠Hadoop Native: Parquet, HDFS and YARN
⢠Extensible - Web Tables, PXF
⢠TPC-DS outperforms Impala by overall 454%
10. SpringXD Interpreter(s)
⢠%xd.stream and %xd.job
⢠Multiple streams or jobs in a paragraph.
⢠Special Deploy/Launch Semantics
⢠Zeppelin Dynamic Forms (${âŚ})
⢠Comprihensive Stream and Job DSL auto-
completion (Ctrl+.)
It simplifies big data projects by orchestrating and automating all steps across multiple data stream pipelinesâcreating, deploying, and managing many pipelines in a unified, extensible, distributed way.
A stream is composed from modules. Each module is deployed to a container and its channels are bound to the transport.
http://10.68.58.121:9995/#/notebook/2BC41KDMZ
add spring xd ambari container layout
%psql.sql
drop table if exists mytable;
create table mytable (i int);
insert into mytable select generate_series(1, 100);
%psql.sql
select * from mytable;
%psql.sql
select count(*) from mytable;
select * from mytable;
add slide showing the HAWQ deployment model (AMBARI)
%psql.sql
drop table if exists mytable;
create table mytable (i int);
insert into mytable select generate_series(1, 100);
%psql.sql
select * from mytable;
%psql.sql
select count(*) from mytable;
select * from mytable;
http://10.68.58.121:8080/#/main/dashboard/metrics
Zeppelin
http://blog.tzolov.net/2015/08/zeppelin-service-for-ambari.html?view=sidebar
https://github.com/tzolov/zeppelin-ambari-plugin
https://bintray.com/big-data/rpm/zeppelin-ambari-plugin/view
Geode (Credits to Steve Shangguan!)
https://github.com/tzolov/ambari-gemfire/tree/geode
https://bintray.com/big-data/rpm/geode-ambari-plugin/view