Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Towards Stateful Serverless

537 Aufrufe

Veröffentlicht am

The Serverless experience is revolutionary and will grow to dominate the future of Cloud Computing. Function-as-a-Service (FaaS) is, however—with its ephemeral, stateless, and short-lived functions—is only the first step. FaaS is great for processing intensive, parallelizable workloads, moving data from A to B providing enrichment and transformation along the way. But it is quite limited and constrained in what use-cases it addresses well, which makes it very hard/inefficient to implement general-purpose application development and distributed systems protocols.

What’s needed is a next-generation Serverless platform and programming model for general-purpose application development—e.g. Fast Data, Reactive Microservices, Streaming Pipelines, ML, etc. What is missing is support for long-lived virtual stateful services, a way to manage distributed state in a scalable and available fashion, ways to physically co-locate data and processing, and options for choosing the right data consistency model for the job.

This talk will discuss the challenges, requirements, and a proposed solution implementing Stateful Serverless with Knative, gRPC, and Akka.

Veröffentlicht in: Ingenieurwesen
  • Als Erste(r) kommentieren

Towards Stateful Serverless

  1. 1. Towards Stateful Serverless Jonas Bonér @jboner
  2. 2. Industry Trends 1. New World: Multicore, Cloud, Mobile, IoT, Big Data, AI 2. Towards Real-time Data-centric Streaming applications 3. Towards a world with automated Operations: Opsless
  3. 3. Reactive Systems The rules of the game Have changed Reactive Manifesto - reactivemanifesto.org
  4. 4. Towards Fast Data Real-time, Data-centric, Event-driven
  5. 5. “We predict that Serverless Computing will grow to dominate the future of Cloud Computing.” - Berkeley CS Department Cloud computing simplified: a Berkeley view on serverless computing
  6. 6. Serverlessprovides a unified solution for Cutting operational & development costs
  7. 7. Serverlessprovides a unified solution for Cutting operational & development costs 1. Cost and resource efficient—scale down to zero 2. Pay as you go—scale up on demand 3. Automation—of scale, failure handling, and recovery 4. Great developer experience—supporting full dev cycle 5. Reduced time, complexity, and risk—when moving to the Cloud
  8. 8. Serverless ≠Faas FaaS = Function-as-a-Service
  9. 9. Faas Why Should we let have all the fun?
  10. 10. what’s Good With FaaS?
  11. 11. what’s Good With FaaS?
  12. 12. what’s Good With FaaS? 1. Paved the way for Serverless
  13. 13. what’s Good With FaaS? 1. Paved the way for Serverless 2. Scaling on-demand from 0 to 10000s request, in a cost-efficient manner
  14. 14. what’s Good With FaaS? 1. Paved the way for Serverless 2. Scaling on-demand from 0 to 10000s request, in a cost-efficient manner 3. Simplifies delivery of scalable and available applications
  15. 15. what’s Good With FaaS? 1. Paved the way for Serverless 2. Scaling on-demand from 0 to 10000s request, in a cost-efficient manner 3. Simplifies delivery of scalable and available applications 4. Encourages good distributed systems design (events-first, loose coupling, etc.)
  16. 16. what’s Good With FaaS? 1. Paved the way for Serverless 2. Scaling on-demand from 0 to 10000s request, in a cost-efficient manner 3. Simplifies delivery of scalable and available applications 4. Encourages good distributed systems design (events-first, loose coupling, etc.) 5. Great as integration layer between various (ephemeral and durable) data sources
  17. 17. what’s Good With FaaS? 1. Paved the way for Serverless 2. Scaling on-demand from 0 to 10000s request, in a cost-efficient manner 3. Simplifies delivery of scalable and available applications 4. Encourages good distributed systems design (events-first, loose coupling, etc.) 5. Great as integration layer between various (ephemeral and durable) data sources 6. Great for stateless & processing-centric workloads
  18. 18. what’s Good With FaaS? 1. Paved the way for Serverless 2. Scaling on-demand from 0 to 10000s request, in a cost-efficient manner 3. Simplifies delivery of scalable and available applications 4. Encourages good distributed systems design (events-first, loose coupling, etc.) 5. Great as integration layer between various (ephemeral and durable) data sources 6. Great for stateless & processing-centric workloads 7. Great as data backbone moving data from A to B, transforming it along the way
  19. 19. Use-cases For FaaS?
  20. 20. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window
  21. 21. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window 1. Low traffic applications—enterprise IT services, and spiky workloads
  22. 22. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window 1. Low traffic applications—enterprise IT services, and spiky workloads 2. Stateless web applications—serving static content form S3 (or similar)
  23. 23. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window 1. Low traffic applications—enterprise IT services, and spiky workloads 2. Stateless web applications—serving static content form S3 (or similar) 3. Embarrassingly parallel processing tasks—invoked on demand & intermittently, examples include: resizing images, object recognition, log analysis
  24. 24. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window 1. Low traffic applications—enterprise IT services, and spiky workloads 2. Stateless web applications—serving static content form S3 (or similar) 3. Embarrassingly parallel processing tasks—invoked on demand & intermittently, examples include: resizing images, object recognition, log analysis 4. Orchestration functions—integration/coordination of calls to third-party services
  25. 25. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window 1. Low traffic applications—enterprise IT services, and spiky workloads 2. Stateless web applications—serving static content form S3 (or similar) 3. Embarrassingly parallel processing tasks—invoked on demand & intermittently, examples include: resizing images, object recognition, log analysis 4. Orchestration functions—integration/coordination of calls to third-party services 5. Composing chains of functions—stateless workflow management, connected via data dependencies
  26. 26. Use-cases For FaaS? Use-cases where throughput is key rather than low latency and requests can be completed in a short time window 1. Low traffic applications—enterprise IT services, and spiky workloads 2. Stateless web applications—serving static content form S3 (or similar) 3. Embarrassingly parallel processing tasks—invoked on demand & intermittently, examples include: resizing images, object recognition, log analysis 4. Orchestration functions—integration/coordination of calls to third-party services 5. Composing chains of functions—stateless workflow management, connected via data dependencies 6. Job scheduling—CRON jobs, triggers, etc.
  27. 27. what’s Bad With FaaS?
  28. 28. what’s Bad With FaaS? Hard to build general-purpose applications
  29. 29. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture
  30. 30. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture 2. Functions are stateless, ephemeral, and short-lived
  31. 31. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture 2. Functions are stateless, ephemeral, and short-lived 3. No direct addressability
  32. 32. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture 2. Functions are stateless, ephemeral, and short-lived 3. No direct addressability 4. No co-location of state and processing
  33. 33. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture 2. Functions are stateless, ephemeral, and short-lived 3. No direct addressability 4. No co-location of state and processing 5. Limited options for managing and coordinating distributed state
  34. 34. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture 2. Functions are stateless, ephemeral, and short-lived 3. No direct addressability 4. No co-location of state and processing 5. Limited options for managing and coordinating distributed state 6. Limited options for modelling various consistency guarantees
  35. 35. what’s Bad With FaaS? Hard to build general-purpose applications 1. Retains the limitations of the 3-tier architecture 2. Functions are stateless, ephemeral, and short-lived 3. No direct addressability 4. No co-location of state and processing 5. Limited options for managing and coordinating distributed state 6. Limited options for modelling various consistency guarantees 7. Limited options for managing durable state, that is scalable and available
  36. 36. • The Serverless Experience—but for general-purpose applications, including modern Fast Data and Reactive systems • Stateful Functions—complementing stateless functions, expanding the toolbox and supported use-cases • The cost efficiencies of FaaS—allowing the user to dial in trade-offs (related to cost, SLOs, use-cases) What We Want
  37. 37. • Training and Serving of Machine Learning Models • Any dynamic in-memory model that needs to build up and served with low latency • Real-time Distributed Stream Processing • E.g. Real-time Prediction/Recommendation Serving, Anomaly Detection • User Sessions, Shopping Carts, Caching • Managing in-memory, yet durable, session state across individual requests • Distributed Resilient Transactional Workflows • Saga Pattern, Workflow Orchestration, Rollback/Compensating Actions • Shared Collaborative Workspaces • Collaborative Document Editing, Blackboards, Chat Rooms, etc. • Leader Election • …and other standard distributed systems patterns/protocols for coordination Support For Use Cases Like
  38. 38. Technical Requirements
  39. 39. 1. Stateful long-lived addressable virtual components—actors Technical Requirements
  40. 40. 1. Stateful long-lived addressable virtual components—actors 2. Wide range of options for distributed coordination & communication patterns Technical Requirements
  41. 41. 1. Stateful long-lived addressable virtual components—actors 2. Wide range of options for distributed coordination & communication patterns 3. Options for managing distributed state reliably at scale— ranging from strong to eventual consistency (durable/ephemeral) Technical Requirements
  42. 42. 1. Stateful long-lived addressable virtual components—actors 2. Wide range of options for distributed coordination & communication patterns 3. Options for managing distributed state reliably at scale— ranging from strong to eventual consistency (durable/ephemeral) 4. Intelligent adaptive placement of stateful functions—physical co-location of state and processing, sharding, and sticky routing Technical Requirements
  43. 43. 1. Stateful long-lived addressable virtual components—actors 2. Wide range of options for distributed coordination & communication patterns 3. Options for managing distributed state reliably at scale— ranging from strong to eventual consistency (durable/ephemeral) 4. Intelligent adaptive placement of stateful functions—physical co-location of state and processing, sharding, and sticky routing 5. Ways of managing end-to-end guarantees and correctness Technical Requirements
  44. 44. 1. Stateful long-lived addressable virtual components—actors 2. Wide range of options for distributed coordination & communication patterns 3. Options for managing distributed state reliably at scale— ranging from strong to eventual consistency (durable/ephemeral) 4. Intelligent adaptive placement of stateful functions—physical co-location of state and processing, sharding, and sticky routing 5. Ways of managing end-to-end guarantees and correctness 6. Predictable performance, latency, and throughput—in startup time, communication/coordination, and storage of data Technical Requirements
  45. 45. User Function Deployment FaaS Abstracting Over Communication
  46. 46. Message In User Function Deployment FaaS Abstracting Over Communication
  47. 47. Message In User Function Deployment Message Out FaaS Abstracting Over Communication
  48. 48. Message In User Function Deployment Message Out FaaS With CRUD
  49. 49. Message In User Function Deployment Database Message Out FaaS With CRUD
  50. 50. Message In User Function Deployment Database Message Out Not Serverless In An Ideal World
  51. 51. Unconstrained database access Makes it hard to Automate operations
  52. 52. “Constraints liberate, liberties constrain.” - Runar Bjarnason
  53. 53. Message In User Function Deployment Message Out Stateful Serverless Abstracting Over State
  54. 54. Message In User Function Deployment Message Out Stateful Serverless Abstracting Over State State In
  55. 55. Message In User Function Deployment Message Out Stateful Serverless Abstracting Over State State In State Out
  56. 56. We Need Better Models For Distributed State
  57. 57. A couple of battle-tested, Yet Constrained, models are: We Need Better Models For Distributed State
  58. 58. A couple of battle-tested, Yet Constrained, models are: We Need Better Models For Distributed State Event Sourcing CRDTs&
  59. 59. Event Sourced Functions Happy Path
  60. 60. Event Sourced Functions Happy Path
  61. 61. Command Event Sourced Functions Happy Path
  62. 62. Command Event Sourced Functions Happy Path
  63. 63. Command Event Sourced Functions Happy Path Command
  64. 64. Command Event Log Event Event Sourced Functions Happy Path Command
  65. 65. Command Event Event Log Event Event Sourced Functions Happy Path Command
  66. 66. Event Sourced Functions Happy Path
  67. 67. SAD Path, RECOVER FROM FAILURE Event Sourced Functions
  68. 68. Event Log SAD Path, RECOVER FROM FAILURE Event Sourced Functions
  69. 69. Event Log REPLAY EventS SAD Path, RECOVER FROM FAILURE Event Sourced Functions
  70. 70. Command In User Function Deployment Reply Out Event Log In Events OUt Serverless Event Sourcing
  71. 71. ACID 2.0
  72. 72. ACID 2.0 Associative Batch-insensitive (grouping doesn't matter) a+(b+c)=(a+b)+c
  73. 73. ACID 2.0 Associative Batch-insensitive (grouping doesn't matter) a+(b+c)=(a+b)+c Commutative Order-insensitive (order doesn't matter) a+b=b+a
  74. 74. ACID 2.0 Associative Batch-insensitive (grouping doesn't matter) a+(b+c)=(a+b)+c Commutative Order-insensitive (order doesn't matter) a+b=b+a Idempotent Retransmission-insensitive (duplication does not matter) a+a=a
  75. 75. Convergent & Commutative Replicated Data Types - Shapiro et. al. 2011 Conflict-Free Replicated Data Types
  76. 76. CRDTACID 2.0 Strong Eventual Consistency Replicated & Decentralized Always Converge Correctly Monotonic Merge Function Highly Available & Scalable Convergent & Commutative Replicated Data Types - Shapiro et. al. 2011 Conflict-Free Replicated Data Types
  77. 77. Data types Counters Registers Sets Maps Graphs (that all compose) CRDTACID 2.0 Strong Eventual Consistency Replicated & Decentralized Always Converge Correctly Monotonic Merge Function Highly Available & Scalable Convergent & Commutative Replicated Data Types - Shapiro et. al. 2011 Conflict-Free Replicated Data Types
  78. 78. Message In User Function Deployment Message Out States/Deltas IN States/deltas OUT Serverless CRDTs
  79. 79. So, What are we Doing About it?
  80. 80. What is Akka?
  81. 81. ✴Cloud Native, Reactive, Distributed Systems Runtime ➡ Implementation of the Actor Model—Concurrency and Distribution ➡ Decentralized, Self-Organizing, Peer-to-peer Service Mesh ➡ Autonomous, Self-healing, Event-driven Services What is Akka?
  82. 82. ✴Cloud Native, Reactive, Distributed Systems Runtime ➡ Implementation of the Actor Model—Concurrency and Distribution ➡ Decentralized, Self-Organizing, Peer-to-peer Service Mesh ➡ Autonomous, Self-healing, Event-driven Services ✴ High Performance, High Throughput, and Low Latency ➡ Communication: Point-to-Point, Pub-Sub, and Streaming ➡ Protocols: HTTP, TCP, Aeron (UDP), Kafka, Reactive Streams, gRPC What is Akka?
  83. 83. ✴Cloud Native, Reactive, Distributed Systems Runtime ➡ Implementation of the Actor Model—Concurrency and Distribution ➡ Decentralized, Self-Organizing, Peer-to-peer Service Mesh ➡ Autonomous, Self-healing, Event-driven Services ✴ High Performance, High Throughput, and Low Latency ➡ Communication: Point-to-Point, Pub-Sub, and Streaming ➡ Protocols: HTTP, TCP, Aeron (UDP), Kafka, Reactive Streams, gRPC ✴ Distributed State Management ➡ CRDTs, Event Sourcing, CQRS ➡ Multi-Datacenter Clustering and Log Replication What is Akka?
  84. 84. ✴Cloud Native, Reactive, Distributed Systems Runtime ➡ Implementation of the Actor Model—Concurrency and Distribution ➡ Decentralized, Self-Organizing, Peer-to-peer Service Mesh ➡ Autonomous, Self-healing, Event-driven Services ✴ High Performance, High Throughput, and Low Latency ➡ Communication: Point-to-Point, Pub-Sub, and Streaming ➡ Protocols: HTTP, TCP, Aeron (UDP), Kafka, Reactive Streams, gRPC ✴ Distributed State Management ➡ CRDTs, Event Sourcing, CQRS ➡ Multi-Datacenter Clustering and Log Replication ✴ Find out more at: akka.io What is Akka?
  85. 85. Serving of Stateful Functions
  86. 86. Knative stateful serving Serving of Stateful Functions
  87. 87. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving Serving of Stateful Functions
  88. 88. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Serving of Stateful Functions User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…)
  89. 89. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Serving of Stateful Functions User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…)
  90. 90. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Serving of Stateful Functions User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) gRPC
  91. 91. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Serving of Stateful Functions User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Distributed Datastore (Cassandra, DynamoDB, Spanner,…) gRPC
  92. 92. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving Knative Events User Function (JavaScript, Go, Java,…) Serving of Stateful Functions User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Distributed Datastore (Cassandra, DynamoDB, Spanner,…) gRPC
  93. 93. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Powered by Akka Cluster Sidecars User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Distributed Datastore (Cassandra, DynamoDB, Spanner,…)
  94. 94. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Powered by Akka Cluster Sidecars User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Akka Sidecar Akka Sidecar Akka Sidecar Distributed Datastore (Cassandra, DynamoDB, Spanner,…)
  95. 95. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Powered by Akka Cluster Sidecars User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Akka Sidecar Akka Sidecar Akka Sidecar Distributed Datastore (Cassandra, DynamoDB, Spanner,…) gRPC
  96. 96. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Powered by Akka Cluster Sidecars User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Akka Sidecar Akka Sidecar Akka Sidecar Akka Cluster Distributed Datastore (Cassandra, DynamoDB, Spanner,…) gRPC
  97. 97. Kubernetes Pod Kubernetes Pod Kubernetes Pod Knative stateful serving User Function (JavaScript, Go, Java,…) Powered by Akka Cluster Sidecars User Function (JavaScript, Go, Java,…) User Function (JavaScript, Go, Java,…) Akka Sidecar Akka Sidecar Akka Sidecar Akka Cluster Distributed Datastore (Cassandra, DynamoDB, Spanner,…) gRPC
  98. 98. 1. The promise of Serverless is revolutionary and could grow to dominate the future of Cloud Computing 2. FaaS is a good first step, but with limited addressable use-cases 3. Serverless 2.0 needs a runtime & programming model 
 for general-purpose application development 4. We have started building it with Knative, Akka, and gRPC 5. We need your help In Summary
  99. 99. bit.ly/stateful-serverless-intro github.com/lightbend/stateful-serverless Get Involved
  100. 100. Thank YouJonas Bonér @jboner

×