Whether it’s connected cars, smart home devices, or industrial applications, IoT applications are rapidly becoming more intelligent. Edge computing is helping lead this transformation as IoT devices not only collect and transmit data, but also perform predictive analytics and respond to local events, even without cloud connectivity. In this session, learn about ML inference at the edge, why it matters, and how to use it to build intelligent IoT applications. Through customer use cases, we demonstrate how to use AWS Greengrass to locate cloud-trained ML models, deploy them to your AWS Greengrass devices, enable access to on-device computing power, and apply the models to locally generated data without connection to the cloud.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Principal Product Manager, IoT Edge & Device Services, Amazon Web Services
SRV201
Push Intelligence to the Edge
Machine Learning on AWS Greengrass Devices
Jason Chen

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS IoT Architecture
Secure device
connectivity
and messaging
Endpoints
AWS IoT Core
Fleet onboarding,
management and
SW updates
Fleet
audit and
protection
IoT data
analytics and
intelligence
Gateway
AWS Greengrass
Things
Sense & Act
Cloud
Storage & Compute
Amazon
Intelligence
Insights & Logic → ActionAWS IoT 1-Click

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How can I extend
AWS intelligence
to the edge?

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data and
state sync
Local
actions
Local
triggers
Security
AWS Greengrass
Extend intelligence to the edge
Local ML
inference
Preview today
Over the air
updates
Protocol
adapter for
OPC-UA
Local
resource
access
Now version 1.5.0

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Greengrass ML Inference
Build and train ML
models in the cloud
Accelerate ML inference
applications on the edge
Devices take action
quickly – even when
disconnected
Use Greengrass to
deploy optimized models
on your target device

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Machine Learning at the Edge
Greengrass ML Inference

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Use Cases
Voice/sound
recognition
Collision
avoidance
Image
recognition
Anomaly
detection
More
!
Smart
Agriculture
Predictive
maintenance
Connected cars Video
surveillance
Robotics

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Benefits of ML Inference at the Edge
Latency Bandwidth Availability Privacy

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who is
Yanmar?
Japanese diesel engine and industrial
equipment manufacturer founded in 1912
Focused on contributing to the realization of
a sustainable society in the coming century
Committed to providing optimal solutions for
customers in food production and harnessing power

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Yanmar’s business challenges
As agriculture is under pressure, Yanmar is committed to
developing transformational greenhouse technology
Seeking to increase
the intelligence of
greenhouse operations
Working to deliver
the right amount of
environmental support
for greenhouse crops
Actively managing plant
growth in greenhouses
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How AWS solves Yammar’s
challengesDeployed AWS Greengrass and Greengrass ML Inference
on the greenhouse camera ecosystem
3G network fees are avoided
Greengrass ML Inference
accesses trained and deployed
ML models to process
greenhouse plant pictures
ML models detect
anomalies and trigger both
alerts and corrective steps
Solves three
main challenges
1
2
3
The IoT-Driven
Precision Agriculture
Solution empowers
farmers to grow crops
by deploying intelligence

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
First Last
Title

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who is
Det
Forenede
Dampskibs
-Selskab
(DFDS)?
European shipping and logistics
company founded in 1866
Company vision is to deliver high performance
and superior reliability for shipping services
and flexible transport solutions
In 2017, DFDS achieved a return on invested
capital of 19%, exceeding last year's result by 1.2%

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Desire to improve energy efficiency and reduce
carbon footprint
It was important for DFDS
to find a solution that could
run locally on ships
DFDS was looking to increase
operational efficiency of ships to
minimize fuel consumption
DFDS’ business challenges
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How AWS solves DFDS’ challenges
Deployed AWS Greengrass ML Inference to
reduce energy consumption for their fleet
ML models predict
ship propulsion using
sensor data
ML models minimize
the propulsion power
on ships
The ML model runs
locally, with
re-training done
in the cloud
The solution
provides
DFDS multiple
benefits
1 2 3 Data is sent
to the cloud,
despite sporadic
connectivity
4 5 Building a data
gold mine, with
potential to run
other projects
Ps/stb Draft RadarInclinometer Shaft Power, Propeller Pitch,
Shaft RPM

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Greengrass ML Inference Overview

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Intelligence on devices
Sense
Generate and receive rich data
about the environment
Infer
Extract relevance from huge
amounts of data in real time
Action
Take smart actions

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why is it hard?
Collect and
moving data
to the cloud
Process
data, build
and train
your model
Deploy
model to the
target device
Build ML
framework
(e.g.,
MXNet) for
different
devices
Write
Inference
app and
deploy it to
the target
device
Utilize
accelerators
such as
GPU

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ML Inference using AWS Greengrass
Train in the cloud
• Massive computing power
• Large repository of data
Trained models
and Lambdas
Extracted
IntelligenceInferences and
take actions
locally on device
AWS Cloud
for training
Inference at the edge
• Low latency
• bandwidth saving
• regulation/privacy
• reliability

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy cloud trained models
to target devices for you
• Locate Amazon SageMaker
trained models in
Greengrass console
• Or bring your own models
• Add your trained model as a
“Machine Learning” resource
to Greengrass group
• Deploy to Greengrass devices
Deploy cloud trained models

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Access on-device ML
accelerators such as
GPU and FPGA from
Lambda functions to
speed up inference
• No code required
• Simply declare the
accelerator as a “Local
Resource” that Lambda
functions need to access
Access hardware accelerators

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Pre-built MXNet and
TensorFlow packages so you
don’t need
build them from scratch
for your devices
• Intel Atom
• NVIDIA Jetson TX2
• Raspberry Pi
You can always bring
your own frameworks
Pre-built MXNet and TensorFlow for devices

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lambda examples
to help you create
inference apps,
showing you how to
• Load trained models
• Apply them to locally
generated data for
local inference
• Take actions
Lambda inference examples

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Benefits of Greengrass ML Inference
Deploy cloud trained
models
Enable
GPU access
Use pre-built
MXNet and
TensoFlow, or bring
your own ML
framework
Lambda
actions

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Summary

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customers and partners

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You!