Deploying and operating Machine Learning models is crucial for manufacturing and in IoT use cases. In this live demo, you’ll see how to build a Structured Streaming application from scratch, ingesting sensor data. You’ll learn how to deploy a machine learning model and integrate it into your Streaming Application to make real-time predictions in your very own virtual factory.

1. Real-Time
Predictions on the
Factory Floor
MATE GULYAS
20MIN
Spark Summit EU

2. Intro

3. INDUSTRY 4.0

4. HISTORY
MECHANISATION
MASS
PRODUCTION
AUTOMATION
SMART
FACTORIES
~1800 1870 1969 2014-

5. EXAMPLES
MASS
CUSTOMISATION

6. EXAMPLES
MASS
CUSTOMISATION

7. EXAMPLES
MASS
CUSTOMISATION

8. EXAMPLES
PREDICTIVE
MAINTENANCE

9. CHALLENGES

10. EXAMPLES
ACTUAL
BIG DATA
200 ms /
32 byte
* 200 sensors
* 20 measurements

11. EXAMPLES
ACTUAL
BIG DATA
0.61 MB / sec
2 GB / hour
360 GB / week
18.7 TB / year
200 ms / 32 byte * 200 sensors * 20 measurements

12. EXAMPLES
200 ms / 32 byte * 200 sensors * 20 measurements
ACTUAL
BIG DATA
18.7 TB / year
5 years retention
93.7 TBfixed dataset size

13. DIVERSE DATA SOURCES
ALSO, IT’S NOT JUST
THE SIZE...

14. LATENCY
100-200ms
100-200ms 10-200ms
200-600ms
TOTAL

15. TECHNOLOGIES

16. EVERY TECHNOLOGY
EXISTS ALREADY!

17. EVERY TECHNOLOGY EXISTS
ALREADY!

18. FOG COMPUTING
EDGE COMPUTING

19. FOG COMPUTING

20. LATENCY
100-200ms
100-200ms 10-200ms
200-600ms
TOTAL

21. LATENCY
15ms
15ms
50 ms
TOTAL
(to response)
200ms
200ms

22. EDGE COMPUTING

24. BEST PRACTICES

25. EDUCATION

26. EDUCATION

27. CURRENT
TECHNOLOGICAL
CHALLENGES

28. MACHINE LEARNING
LIFECYCLE
PROBLEM 1

29. COLLECTING GOOD
DATA
PROBLEM 2

30. STATEFUL
STRUCTURED
STREAMING
PROBLEM 3

31. EXPLAINABLE
MODELS
PROBLEM 4

32. @datapao
datapao.com
linkedin.com/in/gulyasmLinkedin
@gulyasmTwitter
gulyasm@datapao.comEmail
DISCOVER TOGETHER

33. DISCOVER TOGETHER
gulyasm@datapao.comPING ME!

34. THANK YOU!
@datapao
datapao.com