Orit Nissan-Messing talks about boosting AI from development to production with serverless

1. https://nuclio.io
Serverless and AI – SNYC 18
Orit Nissan-Messing
Co-Founder & Chief Architect @ Iguazio
oritn@iguazio.com
nuclio : github.com/nuclio

2. Event-Driven AI: Faster and Smarter Decisions
2
• Immediate insights: sub
seconds from events to data
driven actions
• Smarter decisions: up to
date AI models and fresh data
correlated with historical and
external data in real-time
• Cloud agility: continuous
development and deployment

3. 3
The Real-Time AI Pipeline
Transform
Real-Time DB and Model/File store
Real-time Triggers
and Actions
Enrich
(contextualize)
Infer
(predict)
Real-Time
Sources
Additional Context:
• Historical
• Operational
• Environmental
Act
ML
Model
Lower Time to Action, Higher Throughput

4.  Serverless:
Easy-to-deploy functions, no scripts, Docker files, build, etc.
Auto-scaling and event triggers are built-in
Use any language
 What’s missing?
Slow performance, lack of concurrency
Limited number of event sources (mostly HTTP, some Kafka)
Hard to manage complex functions and dependencies
Using Serverless for the AI Pipeline

5. Nuclio: Open Source Real-time Serverless
• Focus on your app, automate maintenance
• Fast
• High concurrency
• Low latency
• Open
• Runs Everywhere
• Cloud , On-Prem, Hybrid, Laptop

6. 6
 Non-blocking, parallel
 Zero copy, buffer reuse
 Up to 400K
events/sec/process
Nuclio: Taking Serverless to The Next Level
Function
Workers
Event
Listeners
Serverless for compute and data intensive tasks
100x faster than AWS Lambda (really) !
Extreme Performance
Shard 1
Workers
Workers
Shard 2
Shard 3
Shard 4 Workers
Stream and Batch
DB, MQ, File
Functions
 Auto-rebalance, checkpoints
 Any source: Kafka, NATS, Kinesis,
MQTT, event-hub, Iguazio, pub/sub,
RabbitMQ, Cron, cloud-events
 Data bindings
 Shared volumes
 Context cache
Statefullness
Nuclio processor

7. 7
Great Match: Serverless + Kubernetes
DBaaS
Object (S3)
• Cloud independent APIs, on-prem, edge
• Auto-scaling + abstract infrastructure
• Fastest evolving features and community
• One platform for serverless and services
• Abstraction, no Docker files or YAMLs
• Automated dev and ops flow
• Everything is a Kubernetes resource
• “Serverless” data services
• Scalable
• Location agnostic

8. Demo: End-to-end analytics and AI
using Serverless
8

9. 9
Demo: Real-Time Analysis of Financial Data
Tweet
Sentiment
Analysis
Tick feed
Analysis
& Tagging
Real-time Dashboard
News Stream
viewer
World TradingData
Iguazio Unified Real-Time DB Service
SQL | NoSQL | TSDB | Streams | Obj/files
Functions/Model
Dev & Testdeploy
Real-time Analysis

10. 10
Develop
and Train
Deploy
and Run
Consume
Explore
Demo: Real-Time Analysis of Financial Data

11. Thank you
11
See you at the workshop 

12. Demo: Voice Driven Real-Time Analytics
NoSQL APIStream API Object API File API
Security
Queries &
Functions
Unified Data Data Lifecycle
SMART HOME
DEVICE
Voice
Query
SQL API Machine
Learning
Update
Locations
iguazio
Data
Platform
GOOGLE
DIALOGFLO
W
GOOGLE MAP
SERVICE
WEB UI (REACT)
SQL Query
NoSQL
Queries
ELASTIC
APPLICATION
MICRO-SERVICES
(OVER KUBERNETES)
12

13. 13
Open Source Enables Portability and Customization
Same APIs, Same User Experience, Anywhere
With native integration into each cloud platform
Laptop
Cloud
On-Prem or Edge

14. Demo Time
https://www.youtube.com/watch?v=wmRBOhlL-nk

15. 16
 Write code + local testing
 Build code and Docker image
 CI/CD Pipeline
 Add logging and monitoring
 Harden security
 Provision servers + OS
 Handle data/event feed
 Handle failures/auto-scaling
 Handle rolling upgrades
 Configuration management
 Write code + local testing
 Provide spec, push deploy
Traditional Dev & Ops model “Serverless” Development Model
Serverless, Eliminating 80% of The Work
1. Automated by the
serverless platform
2. Pay for what you use
80%

16. Nuclio Open-Source, Real-Time Serverless
Local /
shared image
repo
Pluggable
Data Services
Pluggable
Triggers*
Platform
Controller
Function
…
Event
Listeners
Data
Bindings
Platform Abstraction Layer
Packaged functions, auto-scaling
• Up to 400,000 Events/sec per process, under 0.1 ms latency
• Any trigger
• Any Platform
18