Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Apache Zeppelin 소개

4.765 Aufrufe

Veröffentlicht am

Apache Zeppelin 소개
발표일 : 2015-04-23
발표자 : 이문수 (NFLab)

Veröffentlicht in: Ingenieurwesen
  • Loggen Sie sich ein, um Kommentare anzuzeigen.

Apache Zeppelin 소개

  1. 1. Introduction to Zeppelin Lee moon soo, NFLabs moon@apache.org
  2. 2. me • Lee moon soo, aka. ‘moon’ • moon@apache.org Committer and PPMC of Zeppelin • moon@nflabs.com Co-founder of NFLabs
  3. 3. Agenda • Why do you like Zeppelin? • Why your project likes Zeppelin? • Roadmap • QnA
  4. 4. What is Zeppelin?
  5. 5. Let’s see demo
  6. 6. How Zeppelin started
  7. 7. What gets measured, gets managed - Peter Drucker A price of light is less than the cost of darkness - Arthur C. Nielsen , Founder of ACNielsen War is ninety percent information - Napoleon Bonaparate In God we trust, all others must bring data - W. Edwards Deming Data = Understanding
  8. 8. For data analysis, we need tool
  9. 9. But couldn’t find one i like ImpalaDrill Hive tajoPig Cloudera-ML MLLib MRQL
  10. 10. Decided to make one Really good one
  11. 11. Good analytics environment Analytical language Many Libraries Interactive Visualization Sharing
  12. 12. The First attempt 2012~2013
  13. 13. It’s got graphic REPL, deployment, search, import tool But failed, because
  14. 14. It wasn’t widely used It wasn’t opensource
  15. 15. Second attempt 2013~2014 Opensourced graphic REPL from commercial product The first version of Zeppelin
  16. 16. Second attempt 2013~2014 Not widely used It was slow, difficult to use, …
  17. 17. Third attempt 2014~ After few weeks of study, decided to rewrite Zeppelin Graphic REPl -> Notebook with Apache Spark integration
  18. 18. Third attempt 2014~ Next week, beautifulized
  19. 19. Why do you like Zeppelin?
  20. 20. Web Based web framework d3.js Visualization Language Package management bower Build
  21. 21. Notebook Code Result Code Result 결과가 텝과 뉴라인으로 구분된 테이블 형식이면 자동으로 테이블로 포메팅 Notebook
  22. 22. Notebook Data Visualize
  23. 23. Pivot Pivot
  24. 24. Dynamic Form Creation Dynamic Form
  25. 25. REST Web socket Synchronized Sharing
  26. 26. Zeppelin simplifies data analysis
  27. 27. Why do your project likes Zeppelin?
  28. 28. …. Interpreters Spark PySpark SparkSQL Hive Mysql (JDBC) Markdown Shell Easy to extend
  29. 29. Zeppelin Interpreter Architecture Classloader InterpreterGroup Interpreter Interpreter Server Client … HTTP Rest / Websocket Classloader InterpreterGroup Spark SparkSQL Dep
  30. 30. Classloader InterpreterGroup Interpreter Interpreter Server Client … HTTP Rest / Websocket InterpreterGroup Interpreter Interpreter … Seprate JVM process Thrift Zeppelin Interpreter Architecture
  31. 31. public abstract void open(); public abstract void close(); public abstract InterpreterResult interpret(String st, InterpreterContext context); public abstract void cancel(InterpreterContext context); public abstract int getProgress(InterpreterContext context); public abstract List<String> completion(String buf, int cursor); public abstract FormType getFormType(); public Scheduler getScheduler(); Implementing new Interpreter Must have Good to have More controls
  32. 32. Roadmap • Integration with more distributed processing framework • Flink, Ignite, Tajo, etc.. • Output message streaming • Ability to create rich GUI
  33. 33. ImpalaDrill Hive tajoPig Cloudera-ML MLLib MRQL
  34. 34. QnA
  35. 35. Thanks Lee moon soo moon@apache.org moon@nflabs.com

×