The large O’Reilly survey on serverless adoption indicated that the majority of enterprises have not yet adopted serverless. They have cited the following concerns as main factors: security, the steep learning curve, vendor lock-in, integration/debugging and observability of serverless applications.
In this talk, I will share my views on these concerns and present how Waylay IO has addressed these challenges. Waylay IO’s mission is to finally unlock all promised benefits of serverless computation, with an intuitive and developer-friendly low-code platform.
3. What is Low Code?
Low-code development is a promise that developers can use a platform to code at
a very fast pace, with minimal setup effort and quick deployment.
It reduces the complexity of the application development process.
The basic building block of the Low Code platform should be a small snippet of
code, which like a lego brick is reusable across different use cases.
4. Serverless as a new Cloud paradigm
When lambda hit the mainstream, it was widely accepted that it was the best
candidate for that low code lego brick.
5. Where serverless works well
Data ML pipelines, batch processing, etc … It works and it is great!
8. What is the Industry saying?
Two different impediments hampering serverless adoption:
● Architecture complexity: tracing, observability, debugging, deployment etc...
● Fear: fear of losing control, fear of vendor lock in, fear of weak security,
unpredictable cost etc...
Source: O’Reilly serverless survey, Concerns, what works, and what to expect
10. More than one roadblock, this talk is about the one not in the list!
Source: O’Reilly serverless survey, Concerns, what works, and what to expect
11. Serverless or not?
● Serverless promise:
○ Small snippets of code
○ Infinite scale
○ Separation of responsibilities (experts, IT
folks, Devops, ML folks, solution architects
etc…)
○ Easy to develop - Really?
○ Easy to deploy - Really?
○ Easy to support in production - Really?
● “LAMP” (django, rails, express, laravel …)
○ Easy to implement, deploy and support
○ Use laptop, any IAAS stack or “on prem” HW
18. Functions and entities in serverless?
If functions are “do it” calls who keeps the relation between entities?
● DynamoDB? (NoSQL ...)
● Digital twin? (entity centric)
● GraphQL? (graph centric)
● SPARQL? (ontology, semantics, ….)
● x_SQL? (relationship centric)
24. Finite State Machines orchestrator?
FSM is a stock market in reverse: a small front door entry to a big theater.
You can’t process more than one event at the same time.
FSM for workflows, ticket handling, BPM, response on highly correlated triggers
(events), but not for data stream(s)!
25. Flow engine orchestrator?
Can anyone grasp what is going on here?
Interpretability, decision/control flow, stream merging etc…
26. Batch processing - pipelines
IoT stream IoT Hub Store
IoT Device
● Data
● Commands
● Config
Reports (BI)
Analytics ...
Digital Twin API’s
Other systems
(data)
data
27. Offline analytics approach with Lambda
IoT stream IoT Hub
IoT Device
● Data
● Commands
● Config
ML models
Digital Twin
Batch
processing
Store
triggers
Lambdas
Process flow
API’s
Other systems
(data)
data
28. Stream analytics & Lambda & Orchestrator
IoT stream IoT Hub
Store
IoT Device
● Data
● Commands
● Config
Reports (BI)
Analytics ...
Digital Twin API’s
Other systems
(data)
Stream query/filter
Lambdas
AWS step
functions,
Google Workflow
Rule1 is here!
Rule2 is here!
data
29. Stream processing with Lambda/Flow Engines & Orchestrator
IoT stream IoT Hub
IoT Device
● Data
● Commands
● Config
Digital Twin API’s
Other systems
(data)
Lambdas,
Flow Engines
AWS step
functions,
Google Workflow
Rule is here! How fast?
data
Rule2 is here!
fanout
30. OT/IT Convergence dilemma
● Choice1: Rules with S3/lambda: Lambda fanout,
who knows what is happening? It is great for batch
processing, but not more than that.
● Choice2: Stream analytics (rule 1) followed by
Orchestrator (rule 2): How to add new rules, mix
API’s and other sources?
● Choice3: Orchestrator (Flow) tapping into the
stream directly (queue or not doesn’t matter): How
fast this can be? Who adds rules and to which
particular type of a device? How to add new rules,
mix API’s and streams from different devices and
then use other REST sources?
31. Automation is a star topology
Automation requires constant connections to:
● Stream data
● Time series (historical) data
● Anomaly detection/prediction models
● Meta model (digital twins, relations etc.)
● ERP (IT) systems
● Notifications (email, SMS, calls …)
● ML (REST)
● API (external services)
Data
streams
Historical data
APIs
Predictions
DTwin
Alarms
ERP
Anomalies
36. An “Apple” like developers experience
Waylay IO mission is to bring Apple experiences to the developers community.
37. Waylay’s mission
Waylay IO Low Code Platform
● Low-code portal that simplifies development and deployment
● Easy integration of API enabled services
● AI/ML deployment without tears
● Excellent debugging and observability
● All necessary tools in one place
● From development to production in “no time”
● Built-in state of the art security similar to Auth0
● No setup, nothing to manage
● Pay-Per-Use: you only pay for consumption
38. Waylay supports (and pay for) OPEN source!
Waylay platform is built on top of open source cloud agnostic services, without
dependencies on any specific cloud provider (Open FAAS, Kafka, Cassandra, Elastic,
Mongo, Redis…)
40. Stream processing and lambdas?
Orchestrator can also be a container (in memory) for lambdas, why not?
API (lambda) fanout, that is fine
Inference
“In memory” lambdas
events
DTwin
Streams and DTwin
data - real time
injection
41. Does Lambda need to be an external “RPC” call?
Native ”functions”: 90% ~.02ns execution time. That’s the speed you need for
stream processing! No need to run “external” lambdas + RTT penalty.