This slide is to share what I've learned from the kaggle competition. There 3 topics -1) Overview of the competition 2) Introduction to Decision Tree and 3) R package XGboost.

1. What we got from Red Hat competition
By Umaporn Kerdsaeng
DSTO Knowledge Sharing : 10/27/2016

2. Topics:
1. Overview: Red Hat Competition
2. Introduction to Decision Tree
3. R package for Decision Tree (xgboost)

3. Overview: Red Hat Competition

4. Red Had Competition

5. What is ROC?
• ROC : receiver operating characteristic
• The ROC curve was first developed by electrical engineers and radar
engineers during World War II for detecting enemy objects in battlefields.
• ROC curve is a graphical plot that illustrates the performance of a binary
classifier system as its discrimination threshold is varied.
• The curve is created by plotting the true positive rate (TPR) against the false
positive rate (FPR) at various threshold settings.
https://en.wikipedia.org/wiki/Receiver_operating_characteristic

6. Sensitivity and Specificity
https://www.youtube.com/watch?v=Z5TtopYX1Gc
• True Positive (tp) – Detection
• False Positive (fp) – False alarm
• True Negative (tn)
• False Negative (fn)
• Sensitivity = Probability of Detection
• Specificity = Probability of True Negative
• 1-Specificity = Probability of False alarm

7. Actual outcome distribution
AUC = 0.991725

8. receiver operating characteristic (ROC)
https://www.youtube.com/watch?v=gYIlKUP2hk0
the ROC curve can be generated by
plotting the cumulative distribution
function of the detection probability
in the y-axis versus the cumulative
distribution function of the false-
alarm probability in x-axis.

9. ROC Curve
• https://www.youtube.com/watch?v=OAl6eAyP-yo
Bad Good

10. https://www.youtube.com/watch?v=DiFL-i_zsFg

11. Red Hat Data:

12. Introduction to Decision Tree

13. Introduction to Decision Tree:
https://www.youtube.com/watch?v=eKD5gxPPeY0

14. Introduction to Decision Tree:
https://www.youtube.com/watch?v=eKD5gxPPeY0

15. Introduction to Decision Tree:
https://www.youtube.com/watch?v=eKD5gxPPeY0

16. Introduction to Decision Tree:
https://www.youtube.com/watch?v=eKD5gxPPeY0

17. Introduction to Decision Tree:
https://www.youtube.com/watch?v=eKD5gxPPeY0

18. Introduction to Decision Tree:
https://www.youtube.com/watch?v=eKD5gxPPeY0

19. Introduction to Decision Tree:
https://www.youtube.com/watch?v=AmCV4g7_-QM

20. Introduction to Decision Tree:
https://www.youtube.com/watch?v=AmCV4g7_-QM

21. Introduction to Decision Tree:
https://www.youtube.com/watch?v=nodQ2s0CUbI

22. Introduction to Decision Tree:
https://www.youtube.com/watch?v=AmCV4g7_-QM
Count All 14 5 4 5
Count Yes 9 2 4 3
Count No 5 3 0 2
P+ 0.64 0.40 1 0.60
P- 0.36 0.60 0 0.40
(P+)(log(P+,2) -0.41 -0.53 0 -0.44
(P-)(log(P-,2) -0.53 -0.44 #NUM! -0.53
H(S) 0.94 0.97 0 0.97
weigth 0.36 0.29 0.36
0.94 0.35 0.00 0.35
Gain(S,wind) 0.25
Count All 14 8 6
Count Yes 9 6 3
Count No 5 2 3
P+ 0.64 0.75 0.50
P- 0.36 0.25 0.50
(P+)(log(P+,2) -0.41 -0.31 -0.50
(P-)(log(P-,2) -0.53 -0.50 -0.50
H(S) 0.94 0.81 1.00
weigth 0.57 0.43
0.94 0.46 0.43
Gain(S,wind) 0.0481

23. https://www.youtube.com/watch?v=Q4NVG1IHQOU
Introduction to Decision Tree:

24. A Visual Introduction to Machine Learning
http://www.r2d3.us/visual-intro-to-machine-learning-part-1/

25. R package for Decision Tree
(xgboost)

26. XGBoost: Extreme Gradient Boosting
• An optimized distributed gradient boosting library
• XGBoost only works with numeric vectors. you need to convert all
other forms of data into numeric vectors.
• XGBoost provides a convenient function to do cross (an important
method to measure the model’s prediction power).
• XGBoost can handle missing values in the data

27. XGBoost: Extreme Gradient Boosting
https://www.youtube.com/watch?v=ufHo8vbk6g4
http://blog.nycdatascience.com/faculty/kaggle-winning-solution-xgboost-algorithm-let-us-learn-from-its-author-3/
The minimum information we need to provide is

28. XGBoost: Extreme Gradient Boosting
• Step 1 Load all the libraries
• Step 2 Load the dataset
• Step 4 Tune and Run the model
• Step 3 Data Cleaning & Feature Engineering
• Step 5 Score the Test Population
https://www.analyticsvidhya.com/blog/2016/01/xgboost-algorithm-easy-steps/

29. จิปาถะ
• เรียนรู้เรื่องเดิมๆ ซ้ำๆ รอบหลังๆ จะเข้ำใจมำกขึ้น
• English Knowledge Source
• ไอเดียจะมำแบบไม่เป็นระเบียบ แต่เรำต้องจัดระเบียบควำมคิดและกำรทำงำน
• ลองผิดลองถูกและเรียนรู้ไปพร้อมๆ กัน ต้องลงมือทำ
• จดทุกอย่ำงที่ทำ (พำยเรือวนในอ่ำง)