Weitere ähnliche Inhalte
Ähnlich wie Meetup - Using CloudSearch with DynamoDB (20)
Mehr von Amazon Web Services (20)
Meetup - Using CloudSearch with DynamoDB
- 1. Amazon CloudSearch Meetup
San Francisco
March 28, 2013
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 2. Welcome
! Housekeeping
! Slides will be posted
! Upcoming Events
• April 3, 2013. Webinar 10:00am – 11:00am. "Building Location-Based
Search"
https://www2.gotomeeting.com/register/264455602
• April 30, 2013. AWS Summit, San Francisco.
http://aws.amazon.com/aws-summit-2013/san-francisco/
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 3. Agenda
6:30-7:30 Networking (over Pizza/Beer/Drinks)
7:00-7:30 Using CloudSearch with DynamoDB – Jon Handler,
Amazon CloudSearch Solution Architect
7:30-8:00 EDU 2.0 User Talk - Graham Glass, Founder
8:00-8:30 Getting More from Your Search Queries: How to Tune
Search Requests – Tom Hill, Amazon CloudSearch Solution
Architect
8:30-8:45 SnapGuide User Talk - Sam Kimbrel, Senior Software
Engineer
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 4. Using CloudSearch with DynamoDB
Jon Handler, Amazon CloudSearch Solution Architect
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 5. Do You Want Search With That?
! Amazon Dynamo DB
• High throughput, provisioned latency
• NoSQL model
! Amazon CloudSearch
• Rich query API – free text, fielded, faceting and more
• User-customizable ranking
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 6. Amazon CloudSearch Overview
DNS / Load Balancing AWS Query
Search Domain
Search API Console Doc Command Console Config Command Console
Svc API Line Tools API Line Tools
SEARCH SERVICE DOCUMENT SERVICE CONFIG SERVICE
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 7. Automatic Scaling
DATA Document Quantity and Size
SEARCH INSTANCE SEARCH INSTANCE SEARCH INSTANCE
Index Partition 1 Index Partition 2 Index Partition n
Copy 1 Copy 1 Copy 1
TRAFFIC
Search
Request
Volume and SEARCH INSTANCE SEARCH INSTANCE SEARCH INSTANCE
Complexity Index Partition 1 Index Partition 2 Index Partition n
Copy 2 Copy 2 Copy 2
SEARCH INSTANCE SEARCH INSTANCE SEARCH INSTANCE
Index Partition 1 Index Partition 2 Index Partition n
Copy n Copy n Copy n
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 8. The Big Picture
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 9. Working With DynamoDB and CloudSearch
! Create an Amazon CloudSearch domain
! Upload data from DynamoDB to CloudSearch
! Configure: item attributes map to index fields
! Keep CloudSearch and DynamoDB in sync
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 10. Create An Amazon CloudSearch Domain
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 11. © 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 12. Text fields for
matching user terms
Result enabled to
retrieve source data
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 13. Literal fields for
Faceting
Facet enabled to
retrieve facet counts
Search enabled for
narrowing
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 14. Integer fields for
ranking, narrowing
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 15. Configure the Domain
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 16. Amazon Dynamo DB Data
Primary Key Attributes
id = "tt0076759" title = "Star Wars",
director = "Lucas, George",
genre = {"Action","Adventure","Fantasy","Sci-Fi"},
actor = {"Ford, Harrison","Fisher, Carrie","Hamill, Mark”,
"Jones, James Earl","Guinness, Alec","Johnston, Joe",
"Mayhew, Peter","Cushing, Peter","Prowse, David","Daniels,
Anthony”}
year=1977
id = "tt1411664" title = "Born to Be a Star",
director = "Brady, Tom",
genre = {"Comedy"},
actor = {"Ricci, Christina","Swardson, Nick","Dorff, Stephen”,
"Johnson, Don","Bain, Robin","Herrmann, Edward","Goodman,
Dana”, "Giangrande, Meredith","Dawn, Nadia","Locke,
Tembi","Herschman, Adam"}
year=2011
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 17. CloudSearch SDF
[{"type":"add", !
"id": "tt0076759",!
"version": 1,!
"lang": "en", !
"fields": {!
"title":"Star Wars",!
"director":"Lucas, George",!
"genre": ["Action","Adventure","Fantasy","Sci-Fi"], !
"actor": ["Ford, Harrison","Fisher, Carrie",!
"Hamill, Mark","Jones, James Earl",!
"Guinness, Alec”, ...],!
"year": 1977!
} }, ...]!
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 18. Uploading
! CloudSearch's batch update strategy
! Table scan one row is one document
! Batch multiple rows together and send
! Alternate strategy using Hive/S3/CLTs
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 19. Sending to CloudSearch: Main Loop
def _dynamo_to_cloudsearch(table, domain):!
doc_service = domain.get_document_service()!
for table_item in table.scan():!
docid = table_item['docid']!
version = _cur_timestamp()!
doc_service.add(docid, version,!
_get_fields(table_item))!
if len(doc_service.get_sdf()) >= FOUR_MEGABYTES: !
resp = doc_service.commit()!
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 20. Sending To CloudSearch: Fields
def _get_fields(table_item):!
'''Return a dict with key:value for all!
of the fields in the table'''!
attrs = {} !
for key,value in table_item.iteritems(): !
attrs[key] = !
[item for item in value]!
return attrs !
!
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 21. Limits
! ID sizing
• Dynamo: primary key 2K, range key 1K
• CloudSearch: document id 128 bytes
! Items
• Dynamo: unlimited attributes, up to 64K of data
• CloudSearch: unlimited data, up to 200 fields and 100 field
values
• Attribute/field naming conventions
! Numerics
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 22. Keeping CloudSearch Up-To-Date
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 23. Conclusion
! Combine Amazon CloudSearch and Amazon DynamoDB
• DynamoDB – throughput, durability
• CloudSearch – Rich, powerful search
! Just a few lines of code
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 24. Resources
! Amazon CloudSearch Overview Page
http://aws.amazon.com/cloudsearch/
• FAQs
• Community Forum
• Documentation & Getting Started Tutorial (IMDb)
! Demos and Tutorials
• What Is Amazon CloudSearch
• Introducing Amazon CloudSearch (Features)
• Building a Search Application Using Amazon CloudSearch
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 25. Q&A
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 26. We're Hiring!
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.
- 27. Thank You
Jon Handler / handler@amazon.com
© 2012 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc.