Apache Zeppelin 소개 발표일 : 2015-04-23 발표자 : 이문수 (NFLab)

1. Introduction to Zeppelin
Lee moon soo, NFLabs
moon@apache.org

2. me
• Lee moon soo, aka. ‘moon’
• moon@apache.org Committer and PPMC of Zeppelin
• moon@nflabs.com Co-founder of NFLabs

3. Agenda
• Why do you like Zeppelin?
• Why your project likes Zeppelin?
• Roadmap
• QnA

4. What is Zeppelin?

5. Let’s see demo

6. How Zeppelin started

7. What gets measured, gets managed
- Peter Drucker
A price of light is less than the cost of darkness
- Arthur C. Nielsen , Founder of ACNielsen
War is ninety percent information
- Napoleon Bonaparate
In God we trust, all others must bring data
- W. Edwards Deming
Data = Understanding

8. For data analysis, we need tool

9. But couldn’t find one i like
ImpalaDrill
Hive tajoPig
Cloudera-ML
MLLib
MRQL

10. Decided to make one
Really good one

11. Good analytics environment
Analytical language
Many Libraries
Interactive
Visualization
Sharing

12. The First attempt 2012~2013

13. It’s got graphic REPL, deployment,
search, import tool
But failed, because

14. It wasn’t widely used
It wasn’t opensource

15. Second attempt 2013~2014
Opensourced
graphic REPL
from commercial
product
The first version
of Zeppelin

16. Second attempt 2013~2014
Not widely used
It was slow,
difficult to use,
…

17. Third attempt 2014~
After few weeks of study,
decided to rewrite Zeppelin
Graphic REPl -> Notebook
with Apache Spark
integration

18. Third attempt 2014~
Next week, beautifulized

19. Why do you like Zeppelin?

20. Web
Based web framework
d3.js
Visualization
Language
Package management
bower
Build

21. Notebook
Code
Result
Code
Result
결과가 텝과 뉴라인으로 구분된 테이블 형식이면
자동으로 테이블로 포메팅
Notebook

22. Notebook
Data
Visualize

23. Pivot
Pivot

24. Dynamic Form Creation
Dynamic
Form

25. REST
Web socket
Synchronized
Sharing

26. Zeppelin simplifies data
analysis

27. Why do your project likes
Zeppelin?

28. ….
Interpreters
Spark
PySpark
SparkSQL
Hive
Mysql (JDBC)
Markdown
Shell
Easy to
extend

29. Zeppelin Interpreter Architecture
Classloader
InterpreterGroup
Interpreter Interpreter
Server
Client
…
HTTP Rest / Websocket
Classloader
InterpreterGroup
Spark SparkSQL Dep

30. Classloader
InterpreterGroup
Interpreter Interpreter
Server
Client
…
HTTP Rest / Websocket
InterpreterGroup
Interpreter Interpreter …
Seprate JVM process
Thrift
Zeppelin Interpreter Architecture

31. public abstract void open();
public abstract void close();
public abstract InterpreterResult interpret(String st, InterpreterContext context);
public abstract void cancel(InterpreterContext context);
public abstract int getProgress(InterpreterContext context);
public abstract List<String> completion(String buf, int cursor);
public abstract FormType getFormType();
public Scheduler getScheduler();
Implementing new Interpreter
Must
have
Good to
have
More
controls

32. Roadmap
• Integration with more distributed
processing framework
• Flink, Ignite, Tajo, etc..
• Output message streaming
• Ability to create rich GUI

33. ImpalaDrill
Hive tajoPig
Cloudera-ML
MLLib
MRQL

34. QnA

35. Thanks
Lee moon soo
moon@apache.org
moon@nflabs.com