In this session, Luciano will be walking you through a real use case pipeline that uses Elyra features to help analyze COVID-19 related datasets. He will introduce Elyra, a project built to extend JupyterLab with AI-centric capabilities. He'll showcase the extensions that allow you to build Notebook Pipelines and execute these in a Kubeflow environment, execute notebooks as batch jobs, the ability to create, edit and execute Python scripts directly from JupyterLab

1. Elyra - Extending
JupyterLab for AI
Using Elyra for
COVID-19 Analytics
—
Luciano Resende
IBM - CODAIT

2. About me - Luciano Resende
Open Source AI Platform Architect – IBM – CODAIT
• Senior Technical Staff Member at IBM, contributing to open source for over 10 years
• Currently contributing to : Jupyter Notebook ecosystem, Apache Bahir, Apache
Toree, Apache Spark among other projects related to AI/ML platforms
lresende@us.ibm.com
https://www.linkedin.com/in/lresende
@lresende1975
https://github.com/lresende

3. IBM Open Source
Participation
Learn
Open Source @ IBM
Program touches
78,000
IBMers annually
Consume
Virtually all
IBM products
contain some
open source
• 40,363 pkgs
Per Year
Contribute
• >62K OS Certs
per year
• ~10K IBM
commits per
month
Connect
> 1000
active IBM
Contributors
Working in key OS
projects

4. IBM Open Source
Participation
IBM generated open source innovation
• 137 IBM Open Code projects w/1000+ Github projects
• Projects graduates into full open governance: Node-Red,
OpenWhisk, SystemML, Blockchain fabric among others
• developer.ibm.com/code/open/code/
Community
• IBM focused on 18 strategic communities
• Drive open governance in “Centers of Gravity”
• IBM Leaders drive key technologies and assure freedom of
action
The IBM OS Way is now open sourced
• Training, Recognition, Tooling
• Organization, Consuming, Contributing

5. Center for Open Source
Data and AI
Technologies
5
CODAIT aims to make AI solutions
dramatically easier to create, deploy,
and manage in the enterprise
Relaunch of the Spark Technology
Center (STC) to reflect expanded
mission
CODAIT
http://codait.org
codait (French)
= coder/coded
https://m.interglot.com/fr/en/codait

6. Agenda
- Introduction to the
COVID-19 scenario
- Introduction to Elyra
- Walkthrough the COVID-
19 analytic scenario
exploring the Elyra
features

7. https://fivethirtyeight.com/features/a-comic-strip-tour-of-the-wild-world-of-pandemic-modeling

8. IBM Developer / CODAIT / © 2020 IBM Corporation 8
Leveraging analytics to navigate unprecedented times - Getty
Frederick Reiss
IBM CODAIT
Romeo Kienzler
IBM CODAIT

9. Challenges implementing
COVID-notebooks
9
- How to break apart tasks that are
expensive to run
- Data preparation
- Once data is prepared, analytics can run
multiple times on that same data
- Data updates are frequently
- When data is updated, how to ensure all
tasks are executed on the right order
- How to collaborate and share my
artifacts

10. But what is
Elyra ?

11. Elyra
Elyra is a set of
AI centric extensions for
JupyterLab
Elyra was officially
announced as an open
source project by IBM on
April 29th.
The name Elyra is a word
play with one of the Jupyter
moons “Elara” where we
introduce the “y” from
“Jupyter” to make it “Elyra”

12. JupyterLab
JupyterLab is the next generation
UI for the Jupyter Ecosystem.
Bring all the previous
improvements into a single unified
platform plus more!
Provides a modular, extensible
architecture
Retains backward compatibility
with the old notebook we know
and love

13. JupyterLab
File Explorer
Widgets / Rich Output
Tabbed
Workspaces
Text Editor
Console/Terminal

14. Elyra at GitHub
https://github.com/elyra-ai/elyra
Elyra Documentation
https://elyra.readthedocs.io/en/latest/
Elyra
Elyra is a set of AI centric
extensions to JupyterLab. It
aims to help data scientists,
machine learning engineers
and AI developer’s through the
model development life cycle
complexities.

15. ELYRA
Hybrid Runtime Support
Notebook Pipelines
JupyterLab Extensions
Hybrid runtime support
It simplifies the task of running the notebooks
interactively on cloud machines, improving productivity by
leveraging the power of cloud-based resources
Versioning using git
Simplify tracking changes, enabling better sharing
among teammates
Elyra provides a visual editor for building Notebook-
based AI pipelines, enabling the conversion of
multiple notebooks into batch jobs or workflows.
Notebook Pipelines editor
Notebook as batch jobs
Elyra extends the notebook UI to simplify the
submission of notebooks as a batch job for model
training
Python script execution
Exposes Python Scripts as first-class citizens allowing
users to locally edit their scripts and execute them
against local or cloud-based resources seamlessly.
Fork me at: github.com/elyra-ai

17. Resources
Elyra source code at GitHub
https://github.com/elyra-ai/elyra
Elyra Documentation
https://elyra.readthedocs.io/en/latest/
Elyra announcement and demo video
https://www.youtube.com/watch?v=PuGNijkV5PQ
COVID-19 analytics scenario using Elyra by Fred Reiss
https://www.youtube.com/watch?v=CbcgyzB8c4M&t
STAR
US
&
FORK
US
ON
GITHUB

18. Backup
Slides

19. Pipelines

20. AI / Deep Learning
Workloads
Resource intensive workloads
Requires expensive hardware (GPU, TPU)
Heterogeneous frameworks
Long Running training jobs
– Simple MNIST takes over one hour
WITHOUT a decent GPU
– Other non complex deep learning model
training can easily take over a day WITH
GPUs

21. Training/Deploying Models requires a lot of DevOPS
Model
Serving
Monitoring
Resource
Management
Configuration
Hyperparameter
Optimization
Reproducibility

22. AI / Deep Learning
Workloads Challenges
• How to isolate the training environments to multiple jobs,
based on different deep learning frameworks (and/or
releases) can be submitted/trained on the same time.
• Ability to allocate individual system level resources such as
GPUs, TPUs, etc with different kernels for a period of time.
• Ability to allocate and free up system level resources such as
GPUs, TPUs, etc as they stop being used or when they are idle
for a period of time.

23. AI / Deep Learning
Workloads
Source: https://github.com/Langhalsdino/Kubernetes-GPU-Guide
Containers and Kubernetes Platform
- Containers simplify management of
complicated and heterogenous AI/Deep
Learning infrastructure providing a required
isolation layer to different pods running
different Deep Learning frameworks
- Containers provides a flexible way to deploy
applications and are here to stay
- Kubernetes enables easy management of
containerized applications and resources
with the benefit of Elasticity and Quality of
Services

24. AI Platforms
AI/Deep Learning Platforms aim to
abstract the DevOPS tasks from the
Data Scientist providing a consistent
way to develop AI models independent
of the toolkit/framework being used.

25. Kubeflow
• ML Toolkit for Kubernetes
• Open source and community driven
• Support multiple ML Frameworks
• End-to-end workflows that can be
shared, scaled and deployed

26. Kubeflow Pipelines
Kubeflow Pipelines is a platform for
building and deploying portable,
scalable machine learning (ML)
workflows based on Docker containers.
• End-to-end orchestration: enabling and simplifying the
orchestration of machine learning pipelines.
• Easy experimentation: making it easy for you to try
numerous ideas and techniques and manage your
various trials/experiments.
• Easy re-use: enabling you to re-use components and
pipelines to quickly create end-to-end solutions
without having to rebuild each time.

27. Kubeflow Pipelines
Two key takeaways : A Pipeline and a
Pipeline Component
A pipeline is a description of a machine
learning (ML) workflow, including all of
the workflow components and how they
work together.

28. Kubeflow Pipelines
A pipeline component is an
implementation of a pipeline task.
A component represents a step in the
workflow.

29. Kubeflow Pipelines
Each pipeline component is a container
that contains a program to perform the
task required for that particular step of
your workflow.

30. Model
Training
Model
Validation
Model
Deployment
Source A
(Batch)
Data
Ingestion/Pr
eparation
Source B
(Stream)
Data
Ingestion/Pr
eparation
Source C
(Fetch)
Data
Ingestion/Pr
eparation
Model
Testing
Features
(Snapshot)
Decompose Schedule/Run

31. Kubeflow Pipelines

33. Notebooks
as batch jobs

34. Notebook as batch jobs
Model training can take
hours, if not days.
Elyra extends the
Notebook UI with a new
“submit notebook”
button that simplify the
submission of a single
notebook as a batch job.
Submit Notebook

35. Hybrid Runtime
Support

36. Jupyter Enterprise Gateway website
https://Jupyter.org/enterprise_gateway/
Jupyter Enterprise Gateway source code at GitHub
https://github.com/jupyter/enterprise_gateway
Jupyter Enterprise Gateway Documentation
http://jupyter-enterprise-gateway.readthedocs.io/en/latest/
Supported Kernels
Supported Platforms
Jupyter Enterprise Gateway
Spectrum Conductor
A lightweight, multi-tenant,
scalable and secure gateway
that enables Jupyter
Notebooks to share resources
across an Apache Spark or
Kubernetes cluster for
Enterprise/Cloud use cases
+ +

37. Jupyter Enterprise
Gateway Features
Optimized Resource Allocation
– Utilize resources on all cluster nodes by running kernels
as Spark applications in YARN Cluster Mode.
– Pluggable architecture to enable support for additional
Resource Managers
Enhanced Security
– End-to-End secure communications
- Secure socket communications
- Encrypted HTTP communication using SSL
Multiuser support with user
impersonation
– Enhance security and sandboxing by enabling user
impersonation when running kernels (using Kerberos).
– Individual HDFS home folder for each notebook user.
– Use the same user ID for notebook and batch jobs.
Kernel
Kernel Kernel
Kernel
Kernel
Kernel
Kernel
16
32
48
64
0
20
40
60
80
4 Nodes 8 Nodes 12 Nodes 16 Nodes
xKernels(4GBHeap) Cluster Size (32GB Nodes)
MAXIMUM NUMBER OF
SIMULTANEOUS KERNELS

38. Enterprise Gateway
& Kubernetes
Supported Platforms
Before Jupyter Enterprise Gateway …
- Resources required for all kernels needs to
be allocated during Notebook Server pod
creation
- Resources limited to what is physically
available on the host node that runs all
kernels and associated Spark drivers
After Jupyter Enterprise Gateway …
- Gateway pod very lightweight
- Kernels in their own pod, isolation
- Kernel pods built from community images:
Spark-on-K8s, TensorFlow, Keras, etc.
Before Enterprise Gateway After Enterprise Gateway

39. Bob
Alice
Jupyter
Enterprise
Gateway
Bob
Alice
Container images defined in kernelspec
Community image
Kernel
Spark on Kubernetes
Kernel
Jupyter Enterprise
Gateway - Kubernetes

40. Bob
Alice
Jupyter
Enterprise
Gateway
Bob
Alice
Container images defined in kernelspec
JupyterHub will provision
custom images containing
Notebook + NB2KG
extension
JupyterLab
Jupyter
Notebook
Community image
Kernel
Spark on Kubernetes
Kernel
Jupyter Enterprise Gateway - Kubernetes

41. Python script
execution

42. Python Script
execution
• Create new Python
script from the
workspace launcher
• Navigate trough the
script via Table of
Content outline
• Execute the script
against local or cloud-
based resources
Execute Script Select Environment

43. Git
Integration

44. Git Integration
Elyra provides integrated
support for git
repositories simplifying
tracking changes,
allowing rollback to
working versions of the
code, backups

45. Resources
Elyra source code at GitHub
https://github.com/elyra-ai/elyra
Elyra Documentation
https://elyra.readthedocs.io/en/latest/
Elyra announcement and demo video
https://developer.ibm.com/technologies/artificial-intelligence/blogs/open-source-elyra-ai-toolkit-simplifies-data-model-development/
COVID-19 analytics scenario using Elyra by Fred Reiss
https://www.youtube.com/watch?v=CbcgyzB8c4M&t
STAR
US
&
FORK
US
ON
GITHUB

46. Thank you!