Scalable Cloud Solutions with Node.js

1. Scalable Cloud Solutions with Node.js Aaron Michael König & Mathias Peter Team Centric Software Foto: JD Hancock CC BY 2.0

2. Mathias Peter mp@tcs.de 8+ years experience  JS, Redis, AWS    ! mpneuried Aaron Michael König amk@tcs.de 10+ years experience  JS, iOS, C#    ! exinferis

3. Team Centric Software • Webapps & native mobile apps • Amazon Web Services since 2007 • Redis & Node.js since 2010

4. Node.js since the early days

5. Node.js since the early days • Node.js in production since 0.4

6. Node.js since the early days • Node.js in production since 0.4 • Developed many building blocks in Node.js

7. Node.js since the early days • Node.js in production since 0.4 • Developed many building blocks in Node.js • Many microservices on Node.js

8. Node.js since the early days • Node.js in production since 0.4 • Developed many building blocks in Node.js • Many microservices on Node.js • Released quite a few open source modules over the years

10. Node.js in customer projects

11. Cloud based setup.Electronic ﬁtness devices.

12. Software in a health club? Advanced electronic ﬁtness devices? Foto: JD Hancock CC BY 2.0

13. Software in a health club? Advanced electronic ﬁtness devices? WAT? Foto: JD Hancock CC BY 2.0

14. Training the “old way”

15. Training plan Training the “old way”

16. Training plan Training the “old way”

17. Training plan Training plan Training the “old way”

18. Training the “milon way”

19. Training the “milon way”

20. ☁️ Training the “milon way” Training plan

21. ☁️ Training the “milon way” Training plan

22. milon care device logins

23. Scalability Foto: JD Hancock CC BY 2.0

24. Bilder? Asynchronous TasksSynchronous Tasks Foto: JD Hancock CC BY 2.0

25. Scaling for synchronous tasks • more servers • better hardware • storage • Autoscaling

26. Scaling for synchronous tasks Load Balancer Redis MySQL MySQL S3 DynamoDB EC2 Instances

27. Scaling for synchronous tasks Load Balancer Redis MySQL MySQL S3 DynamoDB EC2 Instances

28. Autoscale Scaling for synchronous tasks Load Balancer Redis MySQL MySQL S3 DynamoDB EC2 Instances

29. Processing Autoscale Scaling for synchronous tasks Load Balancer Redis MySQL MySQL S3 DynamoDB EC2 Instances

30. Scaling for asynchronous tasks Queueing

31. • Logs • Training archiving • Training results • Statistics • House keeping • Achievements Scaling for asynchronous tasks Queueing

32. Advantages of Queueing

33. Advantages of Queueing • Asynchronous Execution • Distribution of work • Reliability • Guaranteed success or failure

34. Queueing … or how a queue should work Foto: JD Hancock CC BY 2.0

36. Queue create queue

37. message D Queue message C message B message A create queueadd messages …

38. message D Queue message C message B message A create queueadd messages … Worker create a worker

39. Worker message D Queue message C message B message Acopy receive and … message A

40. Worker message D Queue message C message B message Acopy receive and … message A process invisible for 30s

41. Worker message D Queue message C message B delete on success message A message Acopy

42. Worker message D Queue message C message B delete on success

43. Worker Queue message D message C message Bcopy receive and … message B

44. Worker Queue message D message C message Bcopy receive and … message B process invisible for 30s

45. Worker Queue message D message C message B message Bcopy

46. Worker Queue message D message C 💩 ERROR ! message B Worker message Bcopy

47. Worker Queue message D message C message B … remain within stack

48. Wait … SCALING ! Foto: JD Hancock CC BY 2.0

49. Queue message D multiple workers 👍 Worker message Ccopymessage C Worker message Bcopymessage B Work er Work er Worker Worker

50. Queue message D multiple workers 👍 Worker message Ccopy message C Worker message Bcopymessage B Work er Work er Worker Worker

51. Redis Simple Message Queue RSMQ made by npm install rsmq

52. Design concept

53. Simple & Fast Only Redis - No management server Guaranteed delivery to a single recipient Visibility timeout for received messages Design concept

54. How to … Foto: JD Hancock CC BY 2.0

56. rsmq.createQueue({qname:"foo", vt:30}, function(err, created){ // ... }); create

57. rsmq.createQueue({qname:"foo", vt:30}, function(err, created){ // ... }); rsmq.sendMessage({qname:"foo", message:"bar"}, function(err){ // ... }); create send

58. rsmq.createQueue({qname:"foo", vt:30}, function(err, created){ // ... }); rsmq.sendMessage({qname:"foo", message:"bar"}, function(err){ // ... }); rsmq.receiveMessage({qname:"foo"}, function(err, message){ // ... { id: "42abc", message: "bar", rc: 1, ... } }); create send receive

59. rsmq.createQueue({qname:"foo", vt:30}, function(err, created){ // ... }); rsmq.sendMessage({qname:"foo", message:"bar"}, function(err){ // ... }); rsmq.receiveMessage({qname:"foo"}, function(err, message){ // ... { id: "42abc", message: "bar", rc: 1, ... } }); create send receive rsmq.deleteMessage({qname:"foo", id: msgid}, function(err, success){ // ... }); delete

60. Foto: JD Hancock CC BY 2.0

61. Foto: JD Hancock CC BY 2.0 npm install rsmq-worker

62. How to … Foto: JD Hancock CC by 2.0

64. var RSMQWorker = require( "rsmq-worker" ); var worker = new RSMQWorker( "foo" ); // poll the "foo" queueinit

65. var RSMQWorker = require( "rsmq-worker" ); var worker = new RSMQWorker( "foo" ); // poll the "foo" queue worker.on("message", function(message, next, msgid){ // ... processing next(); // on error use: next( err ); }); init listen

66. var RSMQWorker = require( "rsmq-worker" ); var worker = new RSMQWorker( "foo" ); // poll the "foo" queue worker.on("message", function(message, next, msgid){ // ... processing next(); // on error use: next( err ); }); worker.start(); init listen start

67. var RSMQWorker = require( "rsmq-worker" ); var worker = new RSMQWorker( "foo" ); // poll the "foo" queue worker.on("message", function(message, next, msgid){ // ... processing next(); // on error use: next( err ); }); worker.start(); init listen start worker.send(„bar"); // send to "foo"send

68. DEMO Foto: JD Hancock CC by 2.0

69. Demo results 1x worker.js 4x worker.js performance > 250%

70. Lightweight  just redis No overhead  trusted environments only, no security Fast  depends on redis Features

71. Want more? Terminal Client RSMQ rest-interface REST Interface For PHP, ASP, Curl … rsmq-cli (under dev.) RSMQ cli Monitoring Usage charts rsmq-monitor (planned) RSMQ monitor Notiﬁcation Engine Email Buffer

72. Questions? Foto: JD Hancock CC by 2.0 Team Centric Software: http://www.tcs.de/ RSMQ: ! http://smrchy.github.io/rsmq/ RSMQ-worker: ! https://github.com/mpneuried/rsmq-worker Demo-Source: ! https://github.com/mpneuried/rsmq-mnug-demo Images: Thanks to JD Hancock for providing CC licensed images Credits

74. RabbitMQ  management server Kue  too complex and not lightweight SQS  only AWS, higher latency Redis-List  we need more than that Why not use …?

Scalable Cloud Solutions with Node.js

Scalable Cloud Solutions with Node.js

Recomendados

Más contenido relacionado

Was ist angesagt?

Was ist angesagt?(20)

Similar a Scalable Cloud Solutions with Node.js

Similar a Scalable Cloud Solutions with Node.js(20)

Último

Último(20)

Scalable Cloud Solutions with Node.js