Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

NoSQL meets Microservices - Michael Hackstein

431 Aufrufe

Veröffentlicht am

Just a few years ago all software systems were designed to be monoliths running on a single big and powerful machine. But nowadays most companies desire to scale out instead of scaling up, because it is much easier to buy or rent a large cluster of commodity hardware then to get a single machine that is powerful enough. In the database area scaling out is realized by utilizing a combination of polyglot persistence and sharding of data. On the application level scaling out is realized by microservices. In this talk I will briefly introduce the concepts and ideas of microservices and discuss their benefits and drawbacks. Afterwards I will focus on the point of intersection of a microservice based application talking to one or many NoSQL databases. We will try and find answers to these questions: Are the differences to a monolithic application? How to scale the whole system properly? What about polyglot persistence? Is there a data-centric way to split microservices?

Veröffentlicht in: Daten & Analysen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

NoSQL meets Microservices - Michael Hackstein

  1. 1. www.arangodb.com NoSQL meets Microservices Michael Hackstein @mchacki
  2. 2. Michael Hackstein ‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features ‣ Host of cologne.js ‣ Master’s Degree
 (spec. Databases and
 Information Systems) 2
  3. 3. Classical Setup 3
  4. 4. Scaling 4 LoadBalancer
  5. 5. Responsibilities 5 LoadBalancer Application Code Consistency Joins
  6. 6. app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; }
  7. 7. memService cpuService app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; }
  8. 8. memService cpuService app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; } server.route({ method: "GET", path: "/mem/{id}", handler: function (request, reply) { reply(dataLookup["key" + request.params.id]); } });
  9. 9. memService cpuService app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; } server.route({ method: "GET", path: "/mem/{id}", handler: function (request, reply) { reply(dataLookup["key" + request.params.id]); } }); server.route({ method: "GET", path: "/cpu", handler: function (request, reply) { reply(cpuWaste()); } });
  10. 10. memService cpuService app Monolith breakdown 6 var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; } server.route({ method: "GET", path: "/mem/{id}", handler: function (request, reply) { reply(dataLookup["key" + request.params.id]); } }); server.route({ method: "GET", path: "/cpu", handler: function (request, reply) { reply(cpuWaste()); } }); server.route({ method: "GET", path: "/app", handler: function (req, reply) { request('http://127.0.0.1:9000/cpu', function (error, response, body) { if (!error && response.statusCode == 200) { request('http://127.0.0.1:9000/mem/' + body, function (error, response, body) { if (!error && response.statusCode == 200) { reply(body); } else { reply({error: error}); } }); } else { reply({error: error}); } }); } });
  11. 11. Introduction to NoSQL 7 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
  12. 12. Polyglot Modeling ‣ If you have structured data ➡ Use a document store ‣ If you have relations between entities and want to efficiently query them by arbitrary steps in between ➡ Use a graph database ‣ If you manage the data structure yourself and do not need complex queries ➡ Use a key-value store ‣ If you have structured data in graph format, or data model might change ➡ Use a multi-model database 8 Use the right tool for the job
  13. 13. Using different databases 9 LoadBalancer
  14. 14. Responsibilities 10 LoadBalancer Application Code Consistency Joins
  15. 15. Responsibilities 10 LoadBalancer Application Code Consistency Joins
  16. 16. Solution 1: Writes only to one master 11
  17. 17. Solution 2: Microservices 12
  18. 18. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  19. 19. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  20. 20. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  21. 21. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  22. 22. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  23. 23. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  24. 24. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  25. 25. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload}) 6 network hops + 2 queries
  26. 26. Live Demo Neo4J + MongoDB 14 https://github.com/mchacki/ms-meets-nosql-examples
  27. 27. Multi Model Database ‣ Can natively store several kinds of data models: ‣ Key-value pairs ‣ Documents ‣ Graphs ‣ Delivers query mechanisms for all data models ‣ ACID transactions ‣ Cross collection joins 15
  28. 28. Using a Multi-Model Database 16 LoadBalancer Application Code Consistency Joins
  29. 29. Using a Multi-Model Database 16 LoadBalancer Application Code Consistency Joins
  30. 30. Foxx ‣ Customized REST API on top of ArangoDB ‣ Microservice framework ‣ Integrate with other microservices ‣ Reuse your Node.js code and NPM modules ‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth ‣ Operations are encapsulated in the database ‣ low network traffic, direct data access ‣ increases data privacy / (~( ) ) /_/ ( _-----_(@ @) ( / /|/--| V " " " " 17
  31. 31. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  32. 32. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  33. 33. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  34. 34. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  35. 35. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  36. 36. Live Demo Foxx 19 https://github.com/mchacki/ms-meets-nosql-examples
  37. 37. ‣ open source and free (Apache 2 license) ‣ sharding & replication ‣ JavaScript throughout (V8 built into server) ‣ drivers for a wide range of languages ‣ web frontend ‣ good & complete documentation ‣ professional as well as community support 20 An overview of other features
  38. 38. Thank you ‣ Further questions? ‣ Follow me on twitter/github: @mchacki ‣ Write me a mail: mchacki@arangodb.com ‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb ‣ Visit us at our booth ‣ Free T-shirts 21

×