The objective of the introductory workshop is to show you how Machine Learning (ML) is get started with building your first ML application. In the process we hope to give you a glimpse of how ML is going to revolutionize the world. This workshop was given in front of 120 Cambodian engineers on 6 December 2017, at the inaugural Startup Jungle session in Phnom Penh. Startup Jungle is a tech startup learning community run by pioneering companies in the Phnom Penh ecosystem, to train for free the next generation of startupers and tech leaders. ### Part 1: Introduction into AI and how to introduce it in your business. Examples of Slash AI projects. Target audience: business and technical :) ### Part 2: Build your first ML application. Target audience: technical :) - Hands-on technical workshop for developers, to show how to build your own neural net step by step. We will apply it to several examples. - Brief introduction into advanced techniques to optimize ML algorithms, training & data sets, classifiers and parsers.

1. AI Machine Learning Workshop
(build your first ML application)
December 6th 2017
STARTUP JUNGLE

2. MACHINE LEARNING 2
TABLE OF CONTENT
Artificial Intelligence introduction
Machine Learning and Neural Networks
Build your first Machine Learning application

3. MACHINE LEARNING 3
TABLE OF CONTENT
Artificial Intelligence introduction
Machine Learning and Neural Networks
Build your first Machine Learning application

4. MACHINE LEARNING 4
Product Foundry.
For innovation teams and ambitious startups/SMEs.
Focus: deep tech.
Venture Builder.
Serial entrepreneurs. Slash Ventures is where we build
our own lean projects.
Offices in Singapore and Phnom Penh

5. MACHINE LEARNING 5
Learning Community.
Learn from the best. Build great products,
and bring them to market.
Meetup.com/Startup-Jungle/
STARTUP JUNGLE

6. MACHINE LEARNING 6
STARTUP JUNGLE
You can ...
1. Join us every month for great (free)
sessions
2. Meet awesome people in tech &
startups
3. Mentor and share your knowledge -
we are RECRUITING great
companies to join!

7. MACHINE LEARNING 7

8. MACHINE LEARNING 8

9. MACHINE LEARNING 9

10. MACHINE LEARNING 1
0

11. MACHINE LEARNING
From Artificial Power . . .
. . . to Artificial Intelligence
11

12. MACHINE LEARNING
WHAT IS ARTIFICIAL INTELLIGENCE?
“the science of engineering and making intelligent
machines”
12

13. MACHINE LEARNING
2 TYPES OF AI
WEAK AI
- Narrow tasks
- Very powerful tool
STRONG AI
- General intelligence
- Outperform humans on all
tasks
13

14. MACHINE LEARNING
2 TYPES OF AI
WEAK AI
- Narrow tasks
- Very powerful tool
STRONG AI
- General intelligence
- Outperform humans on all
tasks
FOCUS OF THIS
TALK
14

15. MACHINE LEARNING
3 WAVES OF AI
15

16. MACHINE LEARNING
WAVE 1: EXPERT SYSTEMS
+ Expert systems are rule-based
- Very narrow, no learning capability, poor in real-world
16

17. MACHINE LEARNING
2004: 0 finish 2005: 5 finish
Limits of expert systems: DARPA car challenge
17

18. MACHINE LEARNING
WAVE 2: MACHINE LEARNING
95% of ‘hyped’ AI applications as of 2017
18

19. MACHINE LEARNING
WAVE 2: MACHINE LEARNING
+ Probabilistic, statistical learning techniques
+ Good at supervised learning, classifying and predicting
- Needs lots of data to learn
- Limited ability to understand context and reason
- Statistically impressive, individually unreliable
19

20. MACHINE LEARNING
WAVE 3: (FUTURE) CONTEXTUAL ADAPTATION
+ AI construct explanatory models
+ Seed AI require much less
training data
20

21. MACHINE LEARNING
EXAMPLE: CAT IMAGE RECOGNITION
21

22. MACHINE LEARNING
3 WAVES OF AI
22

23. MACHINE LEARNING
From AI-assisted Humans . . .
. . . To Human-assisted AI’s
Will Humans evolve into supervisors
of AI’s or bots?
23

24. MACHINE LEARNING
TABLE OF CONTENT
Artificial Intelligence introduction
Machine Learning and Neural Networks
Build your first Machine Learning application
24

25. MACHINE LEARNING
or
DEMO
Cat Dog
25

26. MACHINE LEARNING
HUMAN LEARNING
26

27. MACHINE LEARNING
MACHINE LEARNING
27

28. MACHINE LEARNING
No!!!!!!!!
LOLZ
28

29. MACHINE LEARNING
Learn from experiences Follow instructions
WHAT IS MACHINE LEARNING?
29

30. MACHINE LEARNING
Learn from experiences
DATA
WHAT IS MACHINE LEARNING?
Learn from experiences Follow instructions
30

31. MACHINE LEARNING
BITCOIN PRICE PREDICTION
31

32. MACHINE LEARNING
1007
6416
(100,1007)
(1100,6416)
?
BITCOIN PRICE PREDICTION
32

33. MACHINE LEARNING
1007
6416
(100,1007)
(1100,6416)
?
(Dec,5616)
BITCOIN PRICE PREDICTION
33

34. MACHINE LEARNING
Error :
ERROR CALCULATION
34

35. MACHINE LEARNING
Error :
ERROR CALCULATION
35

36. MACHINE LEARNING
Error :
ERROR CALCULATION
36

37. MACHINE LEARNING
Data : (x , y) , x=input=feature , y = output
f(x) : w*x + b , algorithm
Error : cost function , square error
number of iteration : loop, number of repeat
LINEAR REGRESSION
37

38. MACHINE LEARNING
Ex: we want differentiate between apple and watermelon.
HOW TO TEACH A MACHINE?
or
Apple Watermelon
38

39. MACHINE LEARNING
Simple training data: 2 dimensions (weight, size)
Label
Weight (g) 70 100 1000 1500 1200
Size (cm) 7.6 8.3 31 40 36
IDENTIFY PHYSICAL CHARACTERISTICS OF FRUITS
39

40. MACHINE LEARNING
Label
Weight (g) 70 100 1000 1500 1200
Size (cm) 7.6 8.3 31 40 36
IDENTIFY PHYSICAL CHARACTERISTICS OF FRUITS
40

41. MACHINE LEARNING
● Regression
● Bayesian
● Decision Tree
● Neural Networks
● Deep Learning
● Clustering
MACHINE LEARNING ALGORITHMS
41

42. MACHINE LEARNING
● Regression
● Bayesian
● Decision Tree
● Neural Networks
● Deep Learning
● Clustering
MACHINE LEARNING ALGORITHMS
42

43. MACHINE LEARNING
70
7.6
162.8
Weight
Size
w1= 2
w2= 3
f(x) = Weight * w1 + Size * w2
70 * 2 + 7.6 * 3 = 162.8
100 * 2 + 8.3 * 3 = 224.9
1000 * 2 + 31 * 3 = 2093
1500 * 2 + 40 * 3 = 3120
1200 * 2 + 36 * 3 = 2508
Label
Weight (g) 70 100 1000 1500 1200
Size (cm) 7.6 8.3 31 40 36
NEURAL NETWORK - BEHIND THE SCENE
43

44. MACHINE LEARNING
Label ~ Y
Weight (g) ~X1 70 100 1000 1500 1200
Size (cm) ~ X2 7.6 8.3 31 40 36
f(x) = Weight * w1 + Size * w2
70 * 0.01 + 7.6 * -0.32 = -1.73200
100 * 0.01 + 8.3 * -0.32 = -1.65600
1000 * 0.01 + 31 * -0.32 = 0.08
1500 * 0.01 + 40 * -0.32 = 2.2
1200 * 0.01 + 36 * -0.32 = 0.48
70
7.6
-1.73
w1= 0.01
w2= -0.32
NEURAL NETWORK - BEHIND THE SCENE
Weight
Size
44

45. MACHINE LEARNING
Pixel Values
NEURAL NETWORK - BEHIND THE SCENE
45

46. MACHINE LEARNING
f(x)=y = w1* pixel1 + w2* pixel2 + … + wn * pixeln
255
155
y
pixel1
pixel2
w
1
w
2
55
255
w
3
w
n
pixel3
pixeln
.
.
.
.
.
.
NEURAL NETWORK - BEHIND THE SCENE
46

47. MACHINE LEARNING
SURVEY
Why are you interested in AI?
47

48. MACHINE LEARNING
10min BREAK
Pizzas and beer ☺
48

49. MACHINE LEARNING
TABLE OF CONTENT
Artificial Intelligence introduction
Machine Learning and Neural Networks
Build your first Machine Learning application
49

50. MACHINE LEARNING
TRAINING PROCESS
50

51. MACHINE LEARNING
Weight
Size
W1
W2
Output
Output = Weight * W1+ Size * W2
NEURAL NETWORK ARCHITECTURE
51

52. MACHINE LEARNING
Input
Layer
Hidden
Layer
Output
Layer
Weights
Weights
.
.
.
.
.
.
Pixel 1
Pixel 2
Pixel N
NEURAL NETWORK ARCHITECTURE
52

53. MACHINE LEARNING
● Get better using Data.
● Update Weights.
● 1 Example 1 Prediction.
LEARNINGS SO FAR?
53

54. MACHINE LEARNING
iteration 1:
NN(cat_image) = cat : 0.2, dog : 0.7
iteration 2:
NN(cat_image) = cat : 0.4, dog : 0.5
iteration n :
NN(cat_image) = cat: 0.92, dog:0.05
EXAMPLES
54

55. MACHINE LEARNING
● Classification is the operation that is used in production.
● We are going to create a prediction using those weights.
CLASSIFICATION
55

56. MACHINE LEARNING
MACHINE LEARNING CYCLE
56

57. MACHINE LEARNING
● Epochs (number of iterations)
● Alpha (learning rate)
● Size of the Neural Network.
=> Warning: there is no rule of thumb to tune them.
TUNE WITH HYPERPARAMETERS
57

58. MACHINE LEARNING
● How many times you input the data
● How many times you update the weights
EPOCHS
58

59. MACHINE LEARNING
● By how much you update the weights
● Speed of training
● weight += alpha * update
ALPHA
59

60. MACHINE LEARNING
Iterations : 10 000
alpha : 0.001
1 layer of size 100
Time = 44s
GOOD CONFIGURATION
60

61. MACHINE LEARNING
Iterations : 10 000
alpha : 0.001
1 layer of size 1
Time = 34s
SMALL NETWORK
61

62. MACHINE LEARNING
Iterations : 10 000
alpha : 0.001
5 layers of size 1000
Time = 25mn 25s
BIG NETWORK
62

63. MACHINE LEARNING
Iterations : 10 000
alpha = 0.0000005
1 layer of size 100
LOW ALPHA
63

64. MACHINE LEARNING
Iterations : 10 000
alpha = 0.3
1 layer of size 100
BIG ALPHA
64

65. MACHINE LEARNING
● Training = optimize weights
● Classification = Application.
● Better Training = Better Accuracy.
SUMMARY
65

66. MACHINE LEARNING
MACHINE LEARNING LIBRARIES
66

67. MACHINE LEARNING
Pros:
• High level API based on tensorflow
• Easy for beginners
• Fast results
• Huge documentation
Cons:
• Small community
• Huge documentation
TFLearn
67

68. MACHINE LEARNING
Pros:
● Great Documentation.
● Great feature from dataset to deep learning
algorithm.
● It’s really easy to get into this library.
● Dynamic Graph.
● Debug is easy as it should be.
Cons:
● New library so the community is small.
● Can have bug as it’s quite new.
PyTorch
68

69. MACHINE LEARNING
Classification of sentences in English and Khmer !
TEXT CLASSIFICATION
69

70. MACHINE LEARNING
VISA APPLICATION CHATBOT
A chatbot is a computer program which conducts a conversation
via auditory or textual methods.
70

71. MACHINE LEARNING
HANDWRITTEN DIGITS
71

72. MACHINE LEARNING
● Visa Chatbot
● Speaker Recognition (prototype)
● Speech recognition (prototype)
● Newscrawler
OUR JOURNEY
72

73. MACHINE LEARNING
● No knowledge
● Small objectives
● Read
● Implement
OUR JOURNEY
73

74. MACHINE LEARNING
● Python (But if you already programmed a lot it should be easy)
● Linear Algebra i.e Matrix operation.
● Datastructure
REQUIREMENTS TO START
74

75. MACHINE LEARNING
● http://iamtrask.github.io/2015/07/12/basic-python-network/
● http://iamtrask.github.io/2015/07/27/python-network-part2/
● https://www.youtube.com/watch?v=vOppzHpvTiQ&list=PL2-dafEMk2A7YdKv4XfKpfbTH5z6rEEj3
● http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/
● https://www.coursera.org/learn/machine-learning
● https://ayearofai.com/
- For great free dataset use www.kaggle.com
- Quora.
- Medium.
https://bitbucket.org/ventureslash/neural-network-lib (example to build from scratch a Neural Network)
https://bitbucket.org/ventureslash/naivebayes (example to build from scratch a Naive Bayes)
https://bitbucket.org/ventureslash/demos (demos source code)
USEFUL RESOURCES
75