ML

1. Lecture 1
Introduction
TO
Machine
Learning
By: Amir El-ghamry

2. Course Information
¨ Instructor: Amir EL-Ghamry
¤amir_nabil@mans.edu.eg
¤amirelghamry@email.arizona.edu
¤amirnabilsaleh@gmail.com
¤Facebook @amir.n.saleh
¤Twitter @AmirNabilSaleh
¤Instagram @amirnabilsaleh
(C) Dhruv Batra
2

3. Course Information
¨ About Me:
¤Bachelor of computer science at
Mansoura university @2006
¤Grade : Excellent
¤Rank : First
¤Master degree @ 2012
¤Ph.D. degree @ 2019
(C) Dhruv Batra
3

4. Course Information
¨ About Me:
¤Demonstrator at CS dept 2007 – 2012
¤Assistant Lecturer at CS dept 2012 – 2019
¤Joint supervision mission to USA 2016 – 2018
¤University of Texas at Dallas
4

5. Course Information
¨ About Me:
¤Member of Open Event Data Alliance
(OEDA) Research project at the university of
Texas at Dallas 2017 – 2018
¤ Consultant of Minerva Research project at
Arizona University 2019 – till now
5

6. Syllabus
¨ Basics of Statistical Learning
nLoss functions, bias-variance tradeoff,
overfitting, cross-validation
¨ Supervised Learning
nNearest Neighbour, Naïve Bayes, Logistic
Regression, Support Vector Machines,
Neural Networks, Decision Trees
6

7. Syllabus
¨ Unsupervised Learning
nClustering: k-means
nDimensionality reduction: PCA
¨ Advanced Topics
nConvolution neural network
7

8. Syllabus
¨ You will learn about the methods you
heard about
¨ You will understand algorithms, theory,
applications, and implementations
¨ It’s going to be FUN and HARD WORK J
8

9. Prerequisites
¨ Probability and Statistics
¨ Calculus and Linear Algebra
¨ Algorithms
¨ Programming
¤Python or Your language of choice for project.
¨ Ability to deal with abstract mathematical
concepts
9

10. Textbook
¨ We will have lecture notes.
¨ Reference Books:
¤ [Online]
Machine Learning: A Probabilistic Perspective
Kevin Murphy
¤ [Free PDF from author’s webpage]
Bayesian reasoning and machine learning
David Barber
http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwik
i.php?n=Brml.HomePage
¤ Fundamental of neural network by laurene fausett
10

11. Grading
¨ HomeWorks (%)
¨ Final project (%)
¨ Midterm (%)
¨ Final (%)
¨ Class Participation (%)
11

12. Lets start with Fruit Classification
12
Consider a program that takes image file
as input and output type of fruit

13. Fruit Classification
13
Consider you have two class (Apple - Orange)

14. Traditional programming
14
Manual rule
Compare number of orange and green pixels
And calculate ratio

15. Traditional Programming
15
Computer
Data
Program
Output
Data. : image file
Program : Manual rule (ratio of orange
and green pixels )
Output : is it apple or orange

16. Messy Real World
16
Your rules starts to break . Why
1- What if grey black and white
2- image with No apple or oranges

17. Manual – Rules Solutions
17

18. Manual – Rules Solutions
18

19. Problem of traditional programming
19
Ok – What if a new problem
You need to change the rules
Traditional programming is Not suitable

20. Machine Learning
20
We need an algorithm that figure out the
Rules for us
So we don’t have to write them by hand

21. Machine Learning
21
Computer
Data
Output
Program
Data :
Output :
Program : Rules

22. Classifier
22
A classifier is a function that that some data as
input and assign label (class) to it as output
Fruit classifier
E-mail classifier

23. Learning
23
Apple
Input Output (Label)
Apple
Apple

24. Learning
24

25. Learning Data
25
• Training set
– Used to fit (train) model parameters
• Validation set
– Used to check performance on independent
data and tune parameters
• Test set
– final evaluation of performance after all
parameters fixed

26. Learning Types
26

27. Supervised Learning
27
A Technique that build the classifier Automatically
• Create classifier by finding patterns in image
Collect
training data
Train
Classifier
Make
Predictions

28. Collecting Training Data
28
Take description of fruits image input which is
Called features

29. Collecting Training Data
29
Training Data
• features : Weight , Texture
• Output : Label

30. Collecting Training Data
30
Examples : one raw of the Traing Data

31. Collecting Training Data
31
Note :
The more training data you have
the better classifier you can create .

32. Collecting Training Data
32
Note : you can use the image as a features

33. Changing the Training data
33
Classifying the car based on Horsepower and
number of seats

34. Machine Learning is better
34
• You can create new classifier for new problem
by just changing the training data instead of
Writing rules for each problem
• Reusable

35. Many types of Classifiers
35
• Artificial Neural Network (ANN)
• Support vector machine (SVM)
• Decision tree
• Random forest
• Naïve Bayes
• … etc

36. Good features
36
Classifier is good as the feature you provide
è So
Coming up with good features is one of the
most important jobs in Machine Learning

37. Example - Good feature
37

38. Good features
38
We can use Two features

39. Good features – sample population
39
• Population of 1000 dogs
• Number of greyhounds = 500.
• Number of Labradors = 500.
• We Draw a histogram for their heights
• greyhounds color : red
• Labradors color : blue

40. Good features – sample population
40

41. Good features – sample population
41
If we want to predict the dog with heights
20 , 25 and 35 inches
• Dogs with 20 inches è Labrador
• Dogs with 35 inches è greyhound

42. Good features – sample population
43
• Dog with 25 inches è ????
Probability that it is a greyhound
Or Labrador is very close
That mean:
Height is a useful feature but not
perfect

43. Good features
44
Conclusion :
In Machine Learning we need multiple
features for better results
If you want to know which features to use
Do a thought experiment

44. How many features
45
Other features :
Hair length speed weight

45. Good features – Eye color
46
Height Distribution is equal :

46. Useless features – Eye color
47
• Eye color features tell us nothing regarding
type of dog
• Useless features can affect classifier accuracy

47. Independent features
48
• Independent features gives different type of
information
• Remove highly correlated (redundant)
features

48. Easy to understand features
49
• Imagine we want to Predict how many days
to mail a letter between two different cities

49. Easy to understand features
50
• Easy Feature:
distance between the cities in miles

50. Easy to understand features
51
• Complex Feature:
cities location given by longitude and
latitude
So : Simple relationships are easier to learn

51. Features conclusion
52
ideal features are
1- informative
2- independent
3- simple

52. The End
53
Any questions