1.
How AI connects the dots for IoT:
Augmenting IoT Solutions
with Artificial Intelligence
Danilo Poccia, Technical Evangelist
@danilop

2.
Credit: Gerry Cranham/Fox Photos/Getty Images
http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/

3.
1939 London Underground
Credit: Gerry Cranham/Fox Photos/Getty Images
http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/

4.
1939 London Underground
Credit: Gerry Cranham/Fox Photos/Getty Images
http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/
IoT
Big Data
Machine Learning

5.
One of the big challenges with the IoT is to
Collect Analyze Act on
data from devices to generate insights.

6.
AWS IoT

7.
Customer Success – Philips Healthcare
IoT on AWS to collect and act on critical data across different devices
Philips is a leading health-tech company, working to create a new era of connected and personalized digital health and care..
With the addition of AWS IoT, we will
greatly accelerate the pursuit of our
vision by making it easy to acquire,
process, and act on data from
heterogeneous devices in real time.
• AWS gives Philips customers greater control of their
health with connected digital health solutions that support
healthy living and improved care coordination.
• HealthSuite, powered by AWS, is a digital platform that
manages more than 7 million connected medical-grade,
consumer devices, sensors and mobile apps
• The Philips HealthSuite digital platform analyzes and
stores 15 PB of patient data from 390 million imaging
studies, medical records, and patient inputs.
• AWS provides the reliability, performance and scalability
that Philips needs to help protect patient data which
grows by petabyte/month.
Jeroen Tas
CEO Healthcare Informatics Solutions and Services, Philips
”
“

8.
Customer Success – SONOS
Increase the Value of a Product Over Time with Data Telemetry and Usage Data
All the music on earth, in every room of your home, wirelessly. Sonos is the smart speaker system that streams all
your favorite music to any room, or every room. Control your music with one simple app, and fill your home with
pure, immersive sound.
“A 10 year old product can do things
that hadn’t been invented 10 years
ago. Most importantly, going forward,
people will expect your product to
improve, and if it isn’t being updated
and getting better, you’re literally
being left behind.”
Jon Cotter
Sr. Software Development Manager,
”
“ • Utilizes AWS to collect, store, and process performance
metric data and reports for individual speaker systems.
• Can monitor the quality of speakers in the field, and
dramatically add new functionality to existing speakers
without refreshing hardware.
• Launching Trueplay: a Smart Speaker Tuning services
that measures the acoustics in any room and fine-tunes
your speaker.
• Launching in 2015 yet available to devices purchased
over 5 years ago

9.
“We launched Hive towards the end of
2013 and today we have 75,000
customers… The speed at which we
delivered Hive is directly related to our
decision to use AWS cloud”
Seb Chakraborty
Head of Web and Platform Design
• British Gas started a project called Hive, part of its
Connected Homes Strategy.
• Hive Active Heating allows users to control heating and
hot water remotely from mobile, laptop or
smartphones.
• Such a flexibility allows users to control heating exactly
how they need it and save up to £150 a year on utility
bills
Customer Success – British Gas
Brings central heating control to smartphones with AWS
British Gas uses AWS cloud for scaling the
project and to deliver API toolsets around it.
”
“

10.
IoT Devices Have Limited Resources On Board

11.
Challenges Of Devices Living On The Edge
Round-trip latency
Intermittent connectivity
Expensive bandwidth
Programming and updating embedded software needs specialized skills
Limited to what is on the device unless you rewrite or program the device

12.
AWS Greengrass: Local Compute,
Messaging & Data Caching
Local
compute
Local
data caching
Secure
communications
Local
messaging

13.
Benefits of AWS Greengrass
Respond to local events quickly
Operate offline
Simplified device programming
Reduce the cost of IoT applications

14.
Amazon Greengrass: Example Use Cases
Smart Homes Agriculture Manufacturing

15.
AWS
Snowball

16.
How to maximize the capabilities of IoT devices?
Usually IoT devices have significant limitations in terms of
processing power, memory and interfaces

17.
Artificial Intelligence & Deep Learning At Amazon
Thousands Of Employees Across The Company Focused on AI
Discovery &
Search
Fulfilment &
Logistics
Add ML-powered
features to existing products
Echo &
Alexa

18.
Artificial Intelligence on AWS
P2, F1 &
Elastic GPUs
Deep Learning
AMI and template
Investment in
Apache MXNet

19.
Apache MXNet

20.
Deep Learning Frameworks
MXNet, Caffe, Tensorflow, Theano,
Torch, CNTK and Keras
Pre-installed components to speed
productivity, such as Nvidia drivers, CUDA,
cuDNN, Intel MKL-DNN with MXNet,
Anaconda, Python 2 and 3
AWS Integration
Deep Learning AMI

21.
Amazon AI
Bringing Powerful Artificial Intelligence To All Developers

22.
Amazon Rekognition
Image Recognition And Analysis
Powered By Deep Learning
1

23.
Amazon Rekognition
Deep learning-based image recognition service
Search, verify, and organize millions of images
Object and Scene
Detection
Facial
Analysis
Face
Comparison
Facial
Recognition
Image Moderation

24.
Amazon Rekognition: Images In,
Categories and Facial Analysis Out
Amazon
Rekognition
Car
Outside
Daytime
Driving
Objects
& Scenes
Female
Smiling
Sunglasses
Face ID
DetectLabels
DetectFaces
DetectModerationLabels
CompareFaces
IndexFaces
SearchFacesByImage
Faces

25.
Bynder allows you to easily create, find and use content
for branding automation and marketing solutions.
With our new AI capabilities,
Bynder’s software… now allows
users to save hours of admin
labor when uploading and
organizing their files, adding
exponentially more value.
Chris Hall
CEO, Bynder
”
“
With Rekognition, Bynder revolutionizes marketing admin tasks with AI capabilities

26.
Artfinder is building a world where artists
can make a living doing what they love.
While our artists are able to give
us descriptions, our users may
want to search in ways the artist
hasn’t thought of. We use
Rekognition to obtain objective
labels about an artwork based
on its imagery.
David Tilleyshort
CTO, Artfinder
”
“
With both Rekognition and Amazon ML, Artfinder was able to very rapidly prototype advance
machine learning techniques, with minimum developer time and no upfront infrastructure costs
• A growing catalogue of 300,000
artwork pieces – and visual search a
key part of that.
• Even though Rekognition was
designed to be used for photography,
they have been able to get some
interesting results across their
inventory of paintings, handmade
prints, sculptures, collages, drawings
and photography.

27.
Amazon Polly
Text To Speech Powered By Deep Learning
2

28.
Amazon Polly: Text In, Life-like Speech Out
Amazon Polly
“The temperature
in WA is 75°F”
“The temperature
in Washington is 75 degrees
Fahrenheit”
47 voices spread across 24 languages

29.
TEXT
Market grew by > 20%.
WORDSPHONEMES
{
{
{
{
{
ˈtwɛn.ti
pɚ.ˈsɛnt
ˈmɑɹ.kət ˈgɹu baɪ ˈmoʊɹ
ˈðæn
PROSODY CONTOURUNIT SELECTION AND ADAPTATION
TEXT PROCESSING
PROSODY MODIFICATIONSTREAMING
Market grew by more
than
twenty
percent
Speech units
inventory

30.
aws polly synthesize-speech
--text "It was nice to live such a wonderful live show."
--output-format mp3
--voice-id Joanna
--text-type text
output.mp3

31.
Duolingo voices its language learning service Using Polly
Duolingo is a free language learning service where users
help translate the web and rate translations.
With Amazon Polly our users
benefit from the most lifelike
Text-to-Speech voices
available on the market.
Severin Hacker
CTO, Duolingo
”
“ • Spoken language crucial for
language learning
• Accurate pronunciation matters
• Faster iteration thanks to TTS
• As good as natural human speech

32.
”
“
Royal National Institute of Blind People creates and
distributes accessible information in the form of
synthesized content
Amazon Polly delivers
incredibly lifelike voices which
captivate and engage our
readers.
John Worsfold
Solutions Implementation Manager, RNIB
• RNIB delivers largest library of
audiobooks in the UK for nearly 2 million
people with sight loss
• Naturalness of generated speech is
critical to captivate and engage readers
• No restrictions on speech redistributions
enables RNIB to create and distribute
accessible information in a form of
synthesized content
RNIB provides the largest library in the UK for people with sight loss

33.
Amazon ALEXA
(It’s what’s inside Alexa)
3
Natural Language Understanding (NLU) &
Automatic Speech Recognition (ASR) Powered By Deep Learning

34.
The Advent Of Conversational Interactions
1st Gen: Machine-oriented
interactions

35.
The Advent Of Conversational Interactions
1st Gen: Machine-oriented
interactions
2nd Gen: Control-oriented
& translated

36.
The Advent Of Conversational Interactions
1st Gen: Machine-oriented
interactions
2nd Gen: Control-oriented
& translated
3rd Gen:
Intent-oriented

37.
Amazon Lex: Speech Recognition
& Natural Language Understanding
Amazon Lex
Automatic Speech Recognition
Natural Language Understanding
“What’s the weather
forecast?”
Weather
Forecast

38.
Amazon Lex: Speech Recognition
& Natural Language Understanding
Amazon Lex
Automatic Speech Recognition
Natural Language Understanding
“What’s the weather
forecast?”
“It will be sunny
and 25°C”
Weather
Forecast

39.
Hotel Booking
City New York City
Check In Nov 30th
Check Out Dec 2nd
Hotel Booking
City New York City
Check In
Check Out
“Book a Hotel”
Book Hotel
NYC
“Book a Hotel in
NYC”
Automatic Speech Recognition
Hotel Booking
New York City
Natural Language
Understanding
Intent/Slot
Model
Utterances
“Your hotel is booked
for Nov 30th”
Polly
Confirmation: “Your hotel is
booked for Nov 30th”
a
in
“Can I go ahead with
the booking?”

40.
”
“ Finding missing persons:
~100,000 active missing
persons cases in the U.S.
at any given time
~60% are adults,
~40% are children
• Motorola Solutions applies Amazon
Rekognition, Amazon Polly and Amazon
Lex
• Image analytics and facial recognition
can continually monitor for missing
persons
• Tools that understand natural language
can enable officers to keep eyes up and
hands free
Motorola Solutions is using Amazon AI to help finding missing persons
Motorola Solutions keeps utility workers connected and
visible to each other with real-time voice and data
communication across the smart grid.

41.
I see…
Amazon
Rekognition
Amazon
Polly
Camera
Device
Voice
Synthesize
Speech
Detect Labels
Detect Faces

42.
Nikola Tesla, 1926
“When wireless is perfectly
applied, the whole earth will be
converted into a huge brain…”

43.
Thank you!
Danilo Poccia, Technical Evangelist
@danilop