Apache Toree provides the interactive notebook for Spark/Scala. Toree is a IPython/Jupyter kernel. It lets you mix Spark/Scala code with markdown, execute the notebook, and publish it on the web. Asim will talk about how to install and get started with Apache Toree, how to use it to develop Spark applications interactively in notebooks, and how to publish your notebooks.

1. APACHE TOREE
ASIM JALIS
GALVANIZE

2. INTRO

3. ASIM JALIS
Galvanize/Zipfian, Data
Engineering
Cloudera, Microso!,
Salesforce
MS in Computer Science
from University of
Virginia

4. WHAT IS GALVANIZE’S DATA
ENGINEERING IMMERSIVE?
Immersive Peer Learning
Environment
Master High-Demand
Skills and Technologies
Heart of San Francisco in
SOMA

5. YOU GET TO . . .
Play with Terabytes of
Data
Spark, Hadoop, Hive,
Kafka, Storm, HBase
Data Science at Scale
Level UP your Career

7. FOR MORE INFORMATION
asim.jalis@galvanize.com
http://galvanize.com

8. TALK OVERVIEW

9. WHAT IS THIS TALK ABOUT?
What is Apache Toree?
How can I create
IPython/Jupyter
notebooks for
Spark/Scala?

10. HOW MANY PEOPLE HERE ARE
FAMILIAR WITH
IPYTHON/JUPYTER NOTEBOOKS?

11. HOW MANY PEOPLE HERE ARE
FAMILIAR WITH APACHE SPARK?

12. HOW MANY PEOPLE HERE ARE
FAMILIAR WITH SCALA?

13. LITERATE PROGRAMMING

14. WHAT IS LITERATE
PROGRAMMING?
Proposed by Don Knuth
Write programs for
humans, not machines
Programs communicate
ideas to others
Default text is
documentation or
thoughts
Code is explicitly
marked out

15. LITERATE PROGRAM
This program prints hello world.
<<hello.c>>=
<<includes>>
<<main>>
@
Some includes.
<<includes>>=
#include <stdio.h>
@
Print hello world, then exit.
<<main>>=
int main(int argc, char *argv[]) {
printf("Hello World!n");
return 0;
}
@

16. WHAT PROBLEM DOES JUPYTER
SOLVE?
Suppose you want to share programming idea or tutorial
You write an article and embed code in it
Now imagine being able to execute the code in the article

17. HOW IS THIS DIFFERENT FROM
CODE COMMENTS?
Commented code is not technical literature
It cannot be published or read as an article
A literate program is an executable article

18. JUPYTER/IPYTHON

19. WHAT IS JUPYTER?
Create executable
documents
Originally for Python
Supports other systems
through kernels

20. JUPYTER DEMO
Write Markdown text
Write Scala Spark code
Execute
Repeat
Tab-completion

21. JUPYTER ARCH

22. JUPYTER ARCH
Jupyter server
Displays notebook in
browser
Executes code on
Python runtime
Displays output back
into notebook

23. FERNANDO PÉREZ
IPython/Jupyter
inventor
Particle Physics PhD,
University of Colorado—
Boulder
Now at UC Berkeley
Started IPython in 2001

24. IS IT IPYTHON OR JUPYTER?
Started out as IPython
notebook
Not specific to Python
Can work with Scala and
other languages
Jupyter captures its
language independence

25. APACHE TOREE

26. WHAT IS TOREE?
Toree is a Jupyter Kernel
Executes Scala
Runs Spark
Driver/Context

27. HOW DOES TOREE WORK?

28. TOREE ARCHITECTURE
Jupyter Server talks to
Toree Kernel
Toree Kernel talks to
Spark Driver
Spark Driver talks to
Spark Executors

29. HOW IS TOREE DIFFERENT FROM
ZEPPELIN
Toree compliant with Jupyter protocol
Toree is easy to install and use
Zeppelin does not use Jupyter protocol
Zeppelin wants to be a platform like Jupyter

30. TOREE COMMITS
Lot of activity last year
Stabilizing

31. WHO WROTE TOREE?
Top 2 contributors responsible for 50% of commits
chipsenkbeil has 318 commits
Lull3rSkat3r has 72 commits

32. ROBERT “CHIP” SENKBEIL AND
COREY STUBBS

33. WHY IS IT CALLED TOREE?
Nothing special about the name. Some
people in the group and just picked it out.
Some facts though, it is a purposeful
misspelling of the the Japanese word torii.
A torrii is a traditional gate for Shinto
shrines in Japan. —Corey Stubbs (personal
email)

34. ACTUAL TORII

35. TOREE HANDS-ON DEMO

36. WHAT WE WILL COVER
How to install Toree
How to automatically pull Java libraries in your notebook
How to publish notebooks on
How to turn notebooks into slide shows
http://nbviewer.jupyter.org/

37. QUICKSTART TUTORIAL
For details on how to do all these things
See https://github.com/asimjalis/apache-toree-quickstart

38. CONCLUSION

39. REFERENCES
Apache Toree Quickstart Tutorial
https://github.com/asimjalis/apache-toree-quickstart
Apache Toree on GitHub
https://github.com/apache/incubator-toree
Apache Toree Home
https://toree.incubator.apache.org/

40. QUESTIONS