Weitere Ă€hnliche Inhalte Ăhnlich wie An intro to serverless and OpenWhisk for Kafka users (20) KĂŒrzlich hochgeladen (20) An intro to serverless and OpenWhisk for Kafka users1. © 2019 IBM Corporation
Building serverless apps with Kafka
Dale Lane
An introduction to serverless with Apache OpenWhisk
IBM Event StreamsApache Kafka
@dalelane
2. © 2019 IBM Corporation
Agenda
§ What is âserverlessâ?
§ Implementations
§ Apache OpenWhisk
§ Demos
4. © 2019 IBM Corporation
Serverless
§An architectural pattern where your code is run:
â in response to events
â only on-demand on a per-request basis
5. © 2019 IBM Corporation
Evolution of deployment patterns
Monolith
6. © 2019 IBM Corporation
Evolution of deployment patterns
Microservice
Microservice
Microservice
Monolith
7. © 2019 IBM Corporation
Evolution of deployment patterns
Function
Function
Function
Function
Function
Function
Function
Function
Microservice
Microservice
Microservice
Monolith
8. © 2019 IBM Corporation
Evolution of deployment patterns
Bare metal Virtual
machines
Containers Orchestration Serverless
9. © 2019 IBM Corporation
Function
FaaS
Platform
PaaS
Infrastructure
IaaS
Container
CaaS
Bare metal
Evolution of deployment patterns
10. © 2019 IBM Corporation
Definition
github.com/cncf/wg-serverless
11. © 2019 IBM Corporation
Definition
github.com/cncf/wg-serverless
12. © 2019 IBM Corporation
Definition
applications that do not require server management
applications bundled as one or more functions
[developers] no longer need to spend time and resources on server
provisioning, maintenance, updates, scaling and capacity planning
13. © 2019 IBM Corporation
How is serverless different?
Monolith Microservice
Microservice
Microservice
Function
Function
Function
Function
Function
Function
Function
Function
14. © 2019 IBM Corporation
How is serverless different?
15. © 2019 IBM Corporation
How is serverless different?
16. © 2019 IBM Corporation
How is serverless different?
ÂŁ $ âŹ
kr Fr
„
17. © 2019 IBM Corporation
Example use cases
§ Event processing
§ Data processing
§ Scheduled tasks
§ Scalable REST APIs
19. © 2019 IBM Corporation
Serverless is good for:
§ Infrequent or sporadic demand
§ Focused, short duration processing
§ Highly dynamic workloads
§ Stateless workloads
§ Inconsistent demand
§ Scheduled infrequent tasks
§ Flexibility
21. © 2019 IBM Corporation
Implementations â platform
22. © 2019 IBM Corporation
Implementations â platform
23. © 2019 IBM Corporation
Implementations â platform
24. © 2019 IBM Corporation
Implementations â platform
25. © 2019 IBM Corporation
Implementations â platform
27. © 2019 IBM Corporation
Implementations â private cloud
28. © 2019 IBM Corporation
Implementations â installable
30. © 2019 IBM Corporation
Serverless framework
serverless.com/framework
32. © 2019 IBM Corporation
OpenWhisk
openwhisk.apache.org
37. © 2019 IBM Corporation
OpenWhisk concepts
event action
38. © 2019 IBM Corporation
OpenWhisk concepts
event action
function (eventobj)
{
// do something
return result;
}
41. © 2019 IBM Corporation
OpenWhisk concepts
event action
42. © 2019 IBM Corporation
OpenWhisk concepts
event actiontrigger
43. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
44. © 2019 IBM Corporation
OpenWhisk concepts
actionrule
event
actionrule
actionrule
actionrule
trigger
45. © 2019 IBM Corporation
OpenWhisk concepts
rule
event
rule
rule
rule
trigger
trigger
trigger
trigger
action
event
event
event
46. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
} hello.js
47. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
} hello.js
48. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
}
$ wsk action create helloworld hello.js
ok: created action helloworld
hello.js
49. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
}
$ wsk action create helloworld hello.js
ok: created action helloworld
$ wsk action invoke helloworld
ok: invoked /guest/helloworld with id 7df9c9dfbb
hello.js
50. © 2019 IBM Corporation
OpenWhisk actions
blocking
non-blocking
51. © 2019 IBM Corporation
OpenWhisk actions
blocking
non-blocking
periodic
52. © 2019 IBM Corporation
OpenWhisk actions
action action action
53. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
54. © 2019 IBM Corporation
OpenWhisk implementation
proxy
55. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
56. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
57. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
service registry
invoker
58. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
service registry
invoker
invoker
invoker
invoker
invoker
invoker
59. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
60. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
61. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
62. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
63. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
64. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
65. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
66. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker
67. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
68. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
69. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
70. © 2019 IBM Corporation
Resource limits
github.com/apache/incubator-openwhisk
73. © 2019 IBM Corporation
git clone https://github.com/apache/incubator-openwhisk-devtools.git
cd incubator-openwhisk-devtools/docker-compose
make quick-start
74. © 2019 IBM Corporation
OpenWhisk event providers
trigger ruleevent
75. © 2019 IBM Corporation
cd incubator-openwhisk-devtools/docker-compose
make create-provider-cloudant
make create-provider-kafka
76. © 2019 IBM Corporation
git clone https://github.com/apache/incubator-openwhisk-package-kafka.git
cd incubator-openwhisk-package-kafka
export OPENWHISK_AUTH=$OPENWHISK_HOME/ansible/files/auth.whisk.system
./installKafka.sh $OPENWHISK_AUTH $SYSTEM_IP_ADDR
http://$SYSTEM_IP_ADDR:5984
local_
$SYSTEM_IP_ADDR
78. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
79. © 2019 IBM Corporation
kafka-topics.sh --create --topic MY.EVENTS
--replication-factor 1 --partitions 1
--zookeeper $SYSTEM_IP_ADDR:3181
80. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
81. © 2019 IBM Corporation
wsk trigger create myKafkaTrigger
-f /whisk.system/messaging/kafkaFeed
-p brokers '["9.174.23.208:3092"]'
-p topic MY.EVENTS
82. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
83. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action
84. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action
85. © 2019 IBM Corporation
wsk action create sendToSlack slack-action.js
86. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action
87. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
slackOnKafkaMessages
sendToSlack
action
action
88. © 2019 IBM Corporation
wsk rule create slackOnKafkaMessages
myKafkaTrigger
sendToSlack
89. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
slackOnKafkaMessages
sendToSlack
action
action
92. © 2019 IBM Corporation
Demos â using a hosted service
93. © 2019 IBM Corporation
Demo
Orders Voyages Ships Containers Alerts
VoyagesOrder
Order
Management
Microservice
Fleet/Ships
Simulator
Microservice
Voyages
Management
Microservice
Kafka Streams
Kafka
96. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
âalertsâ
kafkaAlert
trigger
processAlert
action
action
98. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
âalertsâ
kafkaAlert
trigger
processAlert
action
action
99. © 2019 IBM Corporation
Orders Voyages Ships Containers Alerts
VoyagesOrder
Order
Management
Microservice
Fleet/Ships
Simulator
Microservice
Voyages
Management
Microservice
Kafka
Kafka Streams
101. © 2019 IBM Corporation
Summary / roundup
§An architectural pattern where your code is run:
â in response to events
â only on-demand on a per-request basis
102. © 2019 IBM Corporation
Getting started
twitter.com/openwhisk
medium.com/openwhisk
slack.openwhisk.org
youtube.com/channel/UCbzgShnQk8F43NKsvEYA1SA
#openwhisk
developer.ibm.com/code/patterns/category/serverless
serverlessconf.io
103. © 2019 IBM Corporation
Thank you
Dale Lane
Software Engineer, IBM Event Streams
IBM Event Streams - ibm.com/cloud/event-streams
https://slack-invite-ibm-cloud-tech.mybluemix.net/