Introduction to Machine Learning Association Analysis Supervised (inductive) learning Training data includes desired outputs Classification Regression/Prediction Unsupervised learning Training data does not include desired outputs Semi-supervised learning Training data includes a few desired outputs Reinforcement learning Rewards from sequence of actions

1. Machine Learning
CS5122 DESCRIPTIVE & PREDICTIVE ANALYTICS
DILUM BANDARA
Dilum.Bandara@uom.lk
Some slides extracted from CSE 446 Machine Learning by Pedro
Domingos

2. 2
Traditional Programming
Machine Learning
Computer
Data
Program
Output
Computer
Data
Output
Program
Getting computers to program themselves based on data

3. ML in a Nutshell
• 10s of 1000s of machine learning algorithms
• 100s new every year
• Every machine learning algorithm has 3
components:
◦ Representation
◦ Evaluation
◦ Optimization
3

4. Representation
• Sets of rules / Logic programs
• Decision trees
• Instances
• Graphical models (Bayes/Markov nets)
• Neural networks
• Support Vector Machines (SVM)
• Model ensembles
4

5. Evaluation
• Accuracy
• Precision & recall
• Squared error
• Likelihood
• Posterior probability
• Cost / Utility
• Margin
• Entropy
• K-L divergence
5

6. Optimization
• E.g., Greedy search
Combinatorial
optimization
• E.g., Gradient descent
Convex
optimization
• E.g., Linear programming
Constrained
optimization
6

7. Types of Learning
• Association Analysis
• Supervised (inductive) learning
• Training data includes desired outputs
• Classification
• Regression/Prediction
• Unsupervised learning
• Training data does not include desired outputs
• Semi-supervised learning
• Training data includes a few desired outputs
• Reinforcement learning
• Rewards from sequence of actions
7

8. Learning Associations
Basket analysis:
P (Y | X ) probability that somebody who buys X also buys Y
where X and Y are products/services.
Example: P ( milk | beer ) = 0.66
8
Market-Basket transactions
TID Items
1 Bread, Milk
2 Bread, Diaper, Beer, Eggs
3 Milk, Diaper, Beer, Coke
4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke

9. Inductive Learning
Given examples of a function (X, F(X))
Predict function F(X) for new examples X
◦ Discrete F(X): Classification
◦ Continuous F(X): Regression
◦ F(X) = Probability(X): Probability estimation
9

10. Classification
10
 Example: Credit
scoring
 Differentiating
between low-risk and
high-risk customers
from their income and
savings
Discriminant: IF income > θ1 AND savings > θ2
THEN low-risk ELSE high-risk
Model

11. Prediction: Regression
11
 Example: Price of a used car
 x : car attributes
y : price
y = g (x | θ )
g ( ) model,
θ parameters
y = wx+w0

12. Supervised Learning
• Decision tree induction
• Rule induction
• Instance-based learning
• Bayesian learning
• Neural networks
• Support Vector Machines
• Model ensembles
• Learning theory
12

13. Unsupervised Learning
• Clustering
• Dimensionality reduction
13

14. R Examples
• Support Vector Machines
◦ Supervised learning methods
◦ Used for classification & regression tasks
◦ Generates non-overlapping partitions & usually employs
all attributes
• Decision tree
◦ Random forest
14