We do a deeper dive into Amazon Machine Learning, using a specific business problem as an example – predicting if the customer is about to leave your service, also known as customer churn. We examine several practical aspects of building and using a model, including the use of the recipe language for training data manipulation and modeling the costs of false positive/negative errors.

1. Amazon Machine Learning Case Study:
Predic9ng Customer Churn
Denis V. Batalov, Solu9ons Architect, EMEA

2. Customer Churn

3. Machine Learning
Science
• Computer Science
• Sta9s9cs
• Neuroscience
• Opera9ons Research
Ar9ficial Intelligence
• Rule extrac9on from data
• Inspired by human learning
• Adap9ve algorithms
Engineering
• Training: Data à Models
• Predic9on: Models à Forecast
• Decision: Forecast à Ac9ons

4. ML: Robotics

5. ML: Robotics

6. ML: Image Recognition

7. Supervised Learning

8. Supervised Learning
Input Outcome

9. Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome

10. Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
Supervised
Learning
known historical data

11. Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
Supervised
Learning
Unseen Input Same Outcome
known historical data

12. Amazon Machine Learning Service

13. Amazon Machine Learning Service

14. Amazon Machine Learning Service

15. Amazon Machine Learning Service

16. Telco Churn Dataset
• US telco customers, their cell phone plans and usage
• 21 attributes, 3333 rows:
• Customer: State, Area_Code, Phone
• Plan: Intl_Plan, VMail_Plan
• Behavior: VMail_Messages, Day_Mins, Day_Calls,
Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,
Night_Mins, Night_Calls, Night_Charge, Intl_Mins,
Intl_Calls, Intl_Charge
• Other: Account_Length, CustServ_Calls, Churn

17. Telco Churn Dataset
• US telco customers, their cell phone plans and usage
• 21 attributes, 3333 rows:
• Customer: State, Area_Code, Phone
• Plan: Intl_Plan, VMail_Plan
• Behavior: VMail_Messages, Day_Mins, Day_Calls,
Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,
Night_Mins, Night_Calls, Night_Charge, Intl_Mins,
Intl_Calls, Intl_Charge
• Other: Account_Length, CustServ_Calls, Churn

18. Telco Churn Dataset
KS, 128, 415, 382-4657, 0, 1, 25, 265.100000, 110, 45.070000, 197.400000, 99,
16.780000, 244.700000, 91, 11.010000, 10.000000, 3, 2.700000, 1, 0
OH, 107, 415, 371-7191, 0, 1, 26, 161.600000, 123, 27.470000, 195.500000, 103,
16.620000, 254.400000, 103, 11.450000, 13.700000, 3, 3.700000, 1, 0
NJ, 137, 415, 358-1921, 0, 0, 0, 243.400000, 114, 41.380000, 121.200000, 110,
10.300000, 162.600000, 104, 7.320000, 12.200000, 5, 3.290000, 0, 0
OH, 84, 408, 375-9999, 1, 0, 0, 299.400000, 71, 50.900000, 61.900000, 88, 5.260000,
196.900000, 89, 8.860000, 6.600000, 7, 1.780000, 2, 0
OK, 75, 415, 330-6626, 1, 0, 0, 166.700000, 113, 28.340000, 148.300000, 122, 12.610000,
186.900000, 121, 8.410000, 10.100000, 3, 2.730000, 3, 0
AL, 118, 510, 391-8027, 1, 0, 0, 223.400000, 98, 37.980000, 220.600000, 101, 18.750000,
203.900000, 118, 9.180000, 6.300000, 6, 1.700000, 0, 0

19. Creating Datasource for Amazon ML

20. Creating Datasource for Amazon ML

21. Building the Amazon ML Model

22. Recipe
{ "groups": {
"NUMERIC_VARS_NORM":
"group('Intl_Charge','Night_Calls','Day_Calls','Eve_Calls','Eve_Mins','Int
l_Mins','VMail_Message','Intl_Calls','Day_Mins','Night_Mins','Day_Charge',
'Night_Charge','Eve_Charge','Account_Length')” },
"assignments": {},
"outputs": [
"ALL_BINARY",
"State",
"Area_Code",
"normalize(NUMERIC_VARS_NORM)",
"CustServ_Calls"
]
}

23. Recipe: normalize() function
Account_Length Normalized Value
128 0.808771865
107 -0.047574816
137 1.175777586
84 -0.985478323
75 -1.352484044
118 0.400987732

24. Building the Amazon ML Model

25. Cost of Errors
• Cost of Customer Churn and Acquisition (false negative):
• foregone cashflow
• advertising costs
• POS and sign-up admin costs
• Customer Retention Cost (false + true positive)
• Discounts
• Phone upgrades
• etc

26. Financial Outcome of Applying a Model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
False Negative True + False Pos Retention Cost Cost with ML
4.80% 26.40% $100.00 $50.40

27. Financial Outcome of Applying a Model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
False Negative True + False Pos Retention Cost Cost with ML
4.80% 26.40% $100.00 $50.40
• $22.06 of savings per customer
• With 100,000 customers over $2MM in savings with ML

29. @dbatalov