Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Build an AI Virtual Concierge - AWS Summit Sydney

78 Aufrufe

Veröffentlicht am

Business are continuously looking for ways to leverage artifical intelligence to help scale their customer service and support departments. In this session we will step through the process of building a Virtual Concierge experience, powered by Amazon Sumerian, that is able to recognise a visitor at the edge with the AWS DeepLens. You will gain an understanding of the machine learning algorithms that underpin this solution.

  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Build an AI Virtual Concierge - AWS Summit Sydney

  1. 1. S U M M I T SYDNEY
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Build an AI virtual concierge Julian Bright Solutions Architect, ISV Amazon Web Services
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What you’ll get out of this session • An overview of the AI and machine learning services • Why an AI virtual concierge? • An understanding of how facial recognition works • Learn how to build an AI virtual concierge • See live demo
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I TS U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Put machine learning in the hands of every developer Our mission at AWS
  5. 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Some of our machine learning customers…
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The Amazon machine learning stack: broadest and deepest set of capabilities AI SERVICES Easily add intelligence to applications without machine learning skills Vision | Documents | Speech | Language | Chatbots | Forecasting | Recommendations ML SERVICES Build, train, and deploy machine learning models fast and at scale Data labelling | Pre-built algorithms and notebooks | One-click training and deployment ML FRAMEWORKS AND INFRASTRUCTURE Flexibility and choice, highest-performing infrastructure Support for ML frameworks | Compute options purpose-built for machine learning
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The Amazon machine learning stack: broadest and deepest set of capabilities AI SERVICES ML SERVICES Build, train, and deploy machine learning models fast and at scale Data labeling | Pre-built algorithms and notebooks | One-click training and deployment ML FRAMEWORKS AND INFRASTRUCTURE Flexibility and choice, highest-performing infrastructure Support for machine learning frameworks | Compute options purpose-built for ML A M A Z O N R E K O G N I T I O N I M A G E A M A Z O N P O L L Y A M A Z O N T R A N S C R I B E A M A Z O N T R A N S L A T E A M A Z O N C O M P R E H E N D & C O M P R E H E N D M E D I C A L A M A Z O N L E X A M A Z O N R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N F O R E C A S T A M A Z O N T E X T R A C T A M A Z O N P E R S O N A L I S E Language Forecasting Recommendations
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The Amazon machine learning stack: broadest and deepest set of capabilities AI SERVICES ML SERVICES ML FRAMEWORKS AND INFRASTRUCTURE Flexibility and choice, highest-performing infrastructure Support for machine learning frameworks | Compute options purpose-built for ML A M A Z O N R E K O G N I T I O N I M A G E A M A Z O N P O L L Y A M A Z O N T R A N S C R I B E A M A Z O N T R A N S L A T E A M A Z O N C O M P R E H E N D & C O M P R E H E N D M E D I C A L A M A Z O N L E X A M A Z O N R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N F O R E C A S T A M A Z O N T E X T R A C T A M A Z O N P E R S O N A L I S E Language Forecasting Recommendations A M A Z O N S A G E M A K E R B U I L D T R A I N D E P L O Y Pre-built algorithms and notebooks Data labeling (A M A Z O N G R O U N D T R U T H ) Algorithms and models ( A W S M A R K E T P L A C E F O R M A C H I N E L E A R N I N G ) One-click model training and tuning Optimisation (A M A Z O N N E O ) Reinforcement learning One-click deployment & hosting
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The Amazon machine learning stack: broadest and deepest set of capabilities AI SERVICES ML SERVICES ML FRAMEWORKS AND INFRASTRUCTURE A M A Z O N R E K O G N I T I O N I M A G E A M A Z O N P O L L Y A M A Z O N T R A N S C R I B E A M A Z O N T R A N S L A T E A M A Z O N C O M P R E H E N D & C O M P R E H E N D M E D I C A L A M A Z O N L E X A M A Z O N R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N F O R E C A S T A M A Z O N T E X T R A C T A M A Z O N P E R S O N A L I S E Language Forecasting Recommendations A M A Z O N S A G E M A K E R B U I L D T R A I N D E P L O Y Pre-built algorithms and notebooks Data labeling (A M A Z O N G R O U N D T R U T H ) Algorithms and models ( A W S M A R K E T P L A C E F O R M A C H I N E L E A R N I N G ) One-click model training and tuning Optimisation (A M A Z O N N E O ) Reinforcement learning One-click deployment & hosting F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e A M A Z O N E C 2 P 3 & P 3 d n A M A Z O N E C 2 C 5 F P G A s A W S G R E E N G R A S S A M A Z O N E L A S T I C I N F E R E N C E
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The Amazon machine learning stack: broadest and deepest set of capabilities AI SERVICES ML SERVICES ML FRAMEWORKS AND INFRASTRUCTURE A M A Z O N R E K O G N I T I O N I M A G E A M A Z O N P O L L Y A M A Z O N T R A N S C R I B E A M A Z O N T R A N S L A T E A M A Z O N C O M P R E H E N D & C O M P R E H E N D M E D I C A L A M A Z O N L E X A M A Z O N R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N F O R E C A S T A M A Z O N T E X T R A C T A M A Z O N P E R S O N A L I S E Language Forecasting Recommendations A M A Z O N S A G E M A K E R B U I L D T R A I N D E P L O Y Pre-built algorithms and notebooks Data labeling (A M A Z O N G R O U N D T R U T H ) Algorithms and models ( A W S M A R K E T P L A C E F O R M A C H I N E L E A R N I N G ) One-click model training and tuning Optimisation (A M A Z O N N E O ) Reinforcement learning One-click deployment & hosting F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e A M A Z O N E C 2 P 3 & P 3 d n A M A Z O N E C 2 C 5 F P G A s A W S G R E E N G R A S S A M A Z O N E L A S T I C I N F E R E N C E
  11. 11. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Customer reception areas
  13. 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Visitor management technology
  14. 14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  15. 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Virtual concierge experience 1. Recognise visitors 2. Welcome visitor 3. Lookup calendar 4. Contact host 5. Notify visitor
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Virtual concierge experience 1. Recognise visitors 2. Welcome visitor 3. Lookup calendar 4. Contact host 5. Notify visitor
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T How does face recognition work? Object detection Image classification
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Face recognition with machine learning 1 Detect > 2 Align > 3 Represent > 4 Classify https://research.fb.com/publications/deepface-closing-the-gap-to-human-level-performance-in-face-verification/
  19. 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Face recognition probability 99.99% 98%
  20. 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon SageMaker Neo: Train once, run anywhere Neo
  21. 21. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Sagemaker model compilation from sagemaker import get_execution_role, Session from sagemaker.mxnet.model import MXNetModel # Upload pre-trained model role = get_execution_role() model_data = Session().upload_data(path='model.tar.gz', key_prefix='model’) model = MXNetModel(model_data=model_data, role=role, entry_point='predict.py’) Pre-trained model Custom inference code
  22. 22. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Sagemaker model compilation from sagemaker import get_execution_role, Session from sagemaker.mxnet.model import MXNetModel # Upload pre-trained model role = get_execution_role() model_data = Session().upload_data(path='model.tar.gz', key_prefix='model’) model = MXNetModel(model_data=model_data, role=role, entry_point='predict.py’) # Compile for runtime compiled_model = model.compile(job_name='neo', role=role, target_instance_family='ml_m4’, input_shape={'data':[1,3,112,112]}, output_path='compiled’) Choose compile target
  23. 23. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sagemaker model compilation from sagemaker import get_execution_role, Session from sagemaker.mxnet.model import MXNetModel # Upload pre-trained model role = get_execution_role() model_data = Session().upload_data(path='model.tar.gz', key_prefix='model’) model = MXNetModel(model_data=model_data, role=role, entry_point='predict.py’) # Compile for runtime compiled_model = model.compile(job_name='neo', role=role, target_instance_family='ml_m4’, input_shape={'data':[1,3,112,112]}, output_path='compiled’) # Deploy the compiled model predictor = compiled_model.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge') Choose instance type
  24. 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Model deployment for Amazon DeepLens 1. Load pre-trained model in Amazon SageMaker Notebook 2. Compile Amazon SageMaker Neo model to target Amazon DeepLens 3. AWS IOT Greengrass to deploy model and AWS Lambda 4. Face detection at the Edge
  25. 25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Greengrass and AWS Lambda performance # Greengrass infinite loop while True: frame = get_frame() bbox, threshold = detect_face(frame) if threshold > detect_threshold: face = crop(frame, bbox) if classify(face) > sim_threshold: upload_face() draw_frame(bbox, face_name) publish_metrics() 1000 28 22 6 3 1 10 100 1000 Upload Face Detect Face Classify Face Read Frame Publish Metrics Time in milliseconds
  26. 26. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Greengrass and AWS Lambda performance # Greengrass infinite loop while True: frame = get_frame() bbox, threshold = detect_face(frame) if threshold > detect_threshold: face = crop(frame, bbox) if classify(face) > sim_threshold: upload_face() draw_frame(bbox, face_name) publish_metrics() 1000 28 22 6 3 1 10 100 1000 Upload Face Detect Face Classify Face Read Frame Publish Metrics Time in milliseconds Detect face
  27. 27. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Greengrass and AWS Lambda performance # Greengrass infinite loop while True: frame = get_frame() bbox, threshold = detect_face(frame) if threshold > detect_threshold: face = crop(frame, bbox) if classify(face) > sim_threshold: upload_face() draw_frame(bbox, face_name) publish_metrics() Crop and classify face 1000 28 22 6 3 1 10 100 1000 1 2 3 4 5 Time in milliseconds
  28. 28. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Greengrass and AWS Lambda performance # Greengrass infinite loop while True: frame = get_frame() bbox, threshold = detect_face(frame) if threshold > detect_threshold: face = crop(frame, bbox) if classify(face) > sim_threshold: upload_face() draw_frame(bbox, face_name) publish_metrics() 1000 28 22 6 3 1 10 100 1000 Upload Face Detect Face Classify Face Read Frame Publish Metrics Time in milliseconds Upload face to Amazon S3
  29. 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Face registration and detection Amazon DynamoDB Visitors SNS Amazon S3 Visitor Face Amazon API Gateway Register Amazon SageMaker Endpoint 2. Index face Amazon Rekognition Collection 4. Save visitor 7.Face detected SNS Recognise Face Amazon SNS Face Detected AWS DeepLens Amazon S3 Visitor Model Returning Visitor New Visitor 3. Classify face
  30. 30. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  31. 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Virtual concierge experience 1. Recognise visitors 2. Welcome visitor 3. Lookup calendar 4. Contact host 5. Notify visitor
  32. 32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS Step Functions “Serverless” visual workflow management • Coordinate multiple AWS Lambda functions • Handle errors with automatic retry support • Run parallel tasks or activities for callbacks • Logs the state of each step, to help debug Task Choice Handle errors Parallel tasks Start End
  33. 33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Amazon Sumerian as a frontend Instructional design Brand engagement Data visualisation Simulation Virtual showcases Virtual tours Digital signage Customer service agents and chatbots
  34. 34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Receptionist workflow Amazon DynamoDB Appointment Amazon SNS Face Detected AWS API Gateway Notification 5. Get appointment Amazon SNS Email Host 4. Welcome visitor Host Amazon Sumerian Scene Visitor Amazon Rekognition Collection 6. Notify host 7. Host reply AWS Step Function Workflow Amazon DynamoDB Appointment 2. Create session 3. Start workflow
  35. 35. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  36. 36. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Shootsta is all about using technology to enhance our client experience.” Tim Moylan CTO, Shootsta
  37. 37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Summary Registration • Amazon S3 • Amazon Rekognition • Amazon Sagemaker Neo Recognise • AWS IoT Greengrass • AWS DeepLens • Amazon SNS Workflow • AWS Step Functions • AWS Lambda • Amazon DynamoDB Frontend • Amazon Sumerian • Amazon Cognito • Amazon SQS
  38. 38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Call to action Dev Labs: Build an AI Virtual Concierge https://github.com/stephensalim/devlabs-vc Customer and Partner Sumerian exhibits Visy and Olikka Face Recognition with MXNet and Amazon Sagemaker Neo https://github.com/isvlabs/aiml-lab/
  39. 39. Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Julian Bright

×