AWS September Webinar Series - Getting Started with DynamoDB Streams
•Als PPTX, PDF herunterladen•
10 gefällt mir•7,494 views
Modern applications need highly scalable data storage that just works and a scalable way to respond to the continuous pipeline of data changes. DynamoDB Streams provide you with the underlying database infrastructure to create new applications, such as continuously updated free-text search indexes, caches, notifications and change processing for up-to-date analytics. In this webinar we'll give you an overview of DynamoDB Streams, walk through typical use cases, share some sample code, and give you a live demo to get you started building your own amazing applications. Learning Objectives: Understand the benefits of DynamoDB Streams See examples of Streams in real-world scenarios How to get started using DynamoDB Streams Who Should Attend: Developers
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie AWS September Webinar Series - Getting Started with DynamoDB Streams
Ähnlich wie AWS September Webinar Series - Getting Started with DynamoDB Streams (20)
Mehr von Amazon Web Services
Mehr von Amazon Web Services (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
AWS September Webinar Series - Getting Started with DynamoDB Streams
- 1. Event Driven Computing Enabled by DynamoDB Streams Launch update on Cross-region replication and Database Triggers with AWS Lambda Integration
- 2. • What DynamoDB Streams • How • Why DynamoDB Streams
- 3. What are DynamoDB Streams? • Time ordered and partitioned change log • Provides a stream of updates, inserts, deletes • Guaranteed to be delivered only once • Use Kinesis Client Library (KCL), Lambda, or API to query pre- image, post-image, key, timestamp • Scales with your table DynamoDB StreamsDynamoDB
- 6. Cross region replication library • Bootstrapping • Horizontal scaling with KCL Workers • Load balancing • Fault tolerance with check pointing
- 7. Cross region replication Shard 1 Partition 1 Shard 2 Shard 3 Partition 2 KCL Worker KCL Worker KCL Worker
- 8. Cross region replication Partition 1 Partition 2 Partition 3 Partition 4 Partition 5 Shard 1 Shard 2 Shard 3 Shard 4 KCL Worker KCL Worker KCL Worker KCL Worker
- 9. Consuming AWSDynamoDBstreamsAdapterClient adapterClient = new AWSDynamoDBstreamsAdapterClient(updateStreamsCredentials, .. ); .. AmazonDynamoDBClient dynamoDBClient = new AmazonDynamoDBClient(dynamoDBCredentials, ..); .. KinesisClientLibConfiguration workerConfig = new KinesisClientLibConfiguration (.., streamId, updateStreamsCredentials, ..) .withMaxRecords(100) .withInitialPositionInStream(InitialPositionInStream.TRIM_HORIZON); Worker worker = new Worker(recordProcessorFactory, workerConfig, adapterClient, dynamoDBClient, ..); Thread t = new Thread(worker); t.start(); Full code available online
- 10. Processing public class StreamsRecordProcessor implements IRecordProcessor { .. @Override public void processRecords(List<Record> records,.. ) { for(Record record : records) { if (record instanceof RecordAdapter) { Record ddbStreamRecord = ((RecordAdapter) record).getInternalObject(); switch(ddbStreamRecord.getEventName()) { case "INSERT" : case "MODIFY" : DemoHelper.putItem(dynamoDBClient, tableName, ddbStreamRecord.getDynamodb().getNewImage()); break; case ”REMOVE" : DemoHelper.deleteItem(dynamoDBClient, tableName, ddbStreamRecord.getDynamodb().getKeys().get(“Id”).getN()); break; } ...
- 11. DynamoDB Triggers • Trigger AWS Lambda functions • Example – Validate address, send notifications
- 12. DynamoDB Streams and AWS Lambda
- 13. Triggers
- 14. External views
- 18. Partition 1 1000 WCUs Partition K 1000 WCUs Partition M 1000 WCUs Partition N 1000 WCUs Votes Table Candidate A Candidate B Voters Provision 200,000 WCUs
- 19. Candidate A_2 Candidate B_1 Candidate B_2 Candidate B_3 Candidate B_5 Candidate B_4 Candidate B_7 Candidate B_6 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_7 Candidate B_8 Candidate A_6 Candidate A_8 Candidate A_5 Voter Votes Table
- 20. Candidate A_2 Candidate B_1 Candidate B_2 Candidate B_3 Candidate B_5 Candidate B_4 Candidate B_7 Candidate B_6 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_7 Candidate B_8 UpdateItem: “CandidateA_” + rand(0, 10) ADD 1 to Votes Candidate A_6 Candidate A_8 Candidate A_5 Voter Votes Table
- 21. Votes Table Candidate A_2 Candidate B_1 Candidate B_2 Candidate B_3 Candidate B_5 Candidate B_4 Candidate B_7 Candidate B_6 Candidate A_1 Candidate A_3 Candidate A_4 Candidate A_5 Candidate A_6 Candidate A_8 Candidate A_7 Candidate B_8 Periodic Process Candidate A Total: 2.5M 1. Sum 2. Store Voter
- 22. UserId Candidate Date Alice A 2013-10-02 Bob B 2013-10-02 Eve B 2013-10-02 Chuck A 2013-10-02 RawVotes Table Segment Votes A_1 23 B_2 12 B_1 14 A_2 25 AggregateVotes Table Voter 1. Record vote and de-dupe; retry 2. Increment candidate counter
- 23. UserId Candidate Date Alice A 2013-10-02 Bob B 2013-10-02 Eve B 2013-10-02 Chuck A 2013-10-02 RawVotes Table Segment Votes A_1 23 B_2 12 B_1 14 A_2 25 AggregateVotes Table Voter
- 24. AggregateVotes Table Amazon Redshift Amazon EMR Your Amazon Kinesis– Enabled App Voters RawVotes TableVoting App RawVotes DynamoDB Stream
- 25. Real-time voting architecture AggregateVotes Table Amazon Redshift Amazon EMR Your Amazon Kinesis- Enabled App Voters RawVotes TableVoting App RawVotes DynamoDB Stream Handle any scale of election
- 26. Real-time voting architecture AggregateVotes Table Amazon Redshift Amazon EMR Your Amazon Kinesis- Enabled app Voters RawVotes TableVoting App RawVotes DynamoDB Stream Vote only once, no changing votes
- 27. Real-time voting architecture AggregateVotes Table Amazon Redshift Amazon EMR Your Amazon Kinesis– Enabled App Voters RawVotes TableVoting app RawVotes DynamoDB Stream Real-time, fault-tolerant, scalable aggregation
- 28. Real-time voting architecture AggregateVotes Table Amazon Redshift Amazon EMR Your Amazon Kinesis– Enabled App Voters RawVotes TableVoting app RawVotes DynamoDB Stream Voter analytics, statistics
- 30. Op: PUT John Tokyo Op: UPDATE John Pluto Op: UPDATE John Mars DynamoDB writes and Streams Operatio n # DynamoDB Operation Data in DynamoDB Data in Streams 1 PUT : {John:Tokyo} {John:Tokyo} PUT John Tokyo 2 UPDATE: {John:Mars} {John:Mars} UPDATE John Mars 3 UPDATE: {John:Pluto} {John:Pluto} UPDATE John Pluto Data: {Name:Destination}
- 31. View Type Destination Old Image – Before update Name = John, Destination = Mars New Image – After update Name = John, Destination = Pluto Old and New Images Name = John, Destination = Mars Name = John, Destination = Pluto Keys Only Name = John types
- 32. Features of
- 34. Durability & high availability High throughput consensus protocol Replicated across multiple AZs
- 35. Managed
- 38. Durability Records available for 24 hours
- 39. How much does it cost? • Free to turn it on • First 2.5 million reads per month are Free • $0.20 per million reads after that DynamoDB StreamsDynamoDB
- 41. Problem DynamoDB Streams use-case In their own words “
- 42. Problem DynamoDB Streams use-case In their own words “
- 43. The local version of DynamoDB • Desktop Installable • Development & Testing • Publicly available at DynamoDB.com Now supports DynamoDB Streams
- 44. Cross-region replication app: http://tinyurl.com/DynamoDBCrossRegionReplication Open sourced Cross-region library is available http://tinyurl.com/DynamoDBReplicationLibrary
- 45. Thank you!