1. Data Mining Classification: Alternative Techniques Lecture Notes for Chapter 5 Introduction to Data Mining by Tan, Steinbach, Kumar © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004

3. 5.1 Rule-Based Classifier

8. How does Rule-based Classifier Work? R1: (Give Birth = no)  (Can Fly = yes)  Birds R2: (Give Birth = no)  (Live in Water = yes)  Fishes R3: (Give Birth = yes)  (Blood Type = warm)  Mammals R4: (Give Birth = no)  (Can Fly = no)  Reptiles R5: (Live in Water = sometimes)  Amphibians A lemur triggers rule R3, so it is classified as a mammal A turtle triggers both R4 and R5 A dogfish shark triggers none of the rules [Chapter 5 . 1 . 1, page 209]

10. From Decision Trees To Rules Rules are mutually exclusive and exhaustive Rule set contains as much information as the tree [Optional]

11. Rules Can Be Simplified Initial Rule: (Refund=No)  (Status=Married)  No Simplified Rule: (Status=Married)  No [Optional]

17. Example of Sequential Covering [Chapter 5 . 1 . 4, page 214]

18. Example of Sequential Covering… [Chapter 5 . 1 . 4, page 214]

30. Indirect Methods [Chapter 5 . 1 . 5, page 221]

33. Example [Optional]

34. C4.5 versus C4.5rules versus RIPPER C4.5rules: (Give Birth=No, Can Fly=Yes)  Birds (Give Birth=No, Live in Water=Yes)  Fishes (Give Birth=Yes)  Mammals (Give Birth=No, Can Fly=No, Live in Water=No)  Reptiles ( )  Amphibians RIPPER: (Live in Water=Yes)  Fishes (Have Legs=No)  Reptiles (Give Birth=No, Can Fly=No, Live In Water=No)  Reptiles (Can Fly=Yes,Give Birth=No)  Birds ()  Mammals [Chapter 5 . 1 . 5, page 222]

35. C4.5 versus C4.5rules versus RIPPER C4.5 and C4.5rules: RIPPER: [Optional]

38. 5.2 Nearest-Neighbor Classifiers

42. Definition of Nearest Neighbor K-nearest neighbors of a record x are data points that have the k smallest distance to x [Chapter 5 . 2, page 224]

43. 1 nearest-neighbor Voronoi Diagram [Chapter 5 . 2]

50. Example: PEBLS Distance between nominal attribute values : d(Single,Married) = | 2/4 – 0/4 | + | 2/4 – 4/4 | = 1 d(Single,Divorced) = | 2/4 – 1/2 | + | 2/4 – 1/2 | = 0 d(Married,Divorced) = | 0/4 – 1/2 | + | 4/4 – 1/2 | = 1 d(Refund=Yes,Refund=No) = | 0/3 – 3/7 | + | 3/3 – 4/7 | = 6/7 [Optional] 1 4 2 No 1 0 2 Yes Divorced Married Single Marital Status Class 4 3 No 3 0 Yes No Yes Refund Class

51. Example: PEBLS Distance between record X and record Y: where: w X  1 if X makes accurate prediction most of the time w X > 1 if X is not reliable for making predictions [Optional]

52. 5.3 Bayesian Classifiers

63. Example of Naïve Bayes Classifier A: attributes M: mammals N: non-mammals P(A|M)P(M) > P(A|N)P(N) => Mammals [Optional]

65. 5.4 Artificial Neural Network (ANN)

66. Artificial Neural Networks (ANN) Output Y is 1 if at least two of the three inputs are equal to 1. [Chapter 5 . 4, page 247]

67. Artificial Neural Networks (ANN) [Chapter 5 . 4]

69. General Structure of ANN Training ANN means learning the weights of the neurons [Chapter 5 . 4 . 2, page 251]

71. 5.5 Support Vector Machine (SVM)

78. Support Vector Machines [Chapter 5 . 5 ]

84. 5.6 Ensemble Methods

86. General Idea [Chapter 5 . 6 . 1, page 278]

94. Illustrating AdaBoost [Chapter 5 . 6 . 5 ] Data points for training Initial weights for each data point

95. Illustrating AdaBoost [Chapter 5 . 6 . 5 ]