Amazon Machine Learning Case Study:
Predic9ng Customer Churn
Denis V. Batalov, Solu9ons Architect, EMEA

Customer Churn

Machine Learning
Science
• Computer Science
• Sta9s9cs
• Neuroscience
• Opera9ons Research
Ar9ficial Intelligence
• Rule extrac9on from data
• Inspired by human learning
• Adap9ve algorithms
Engineering
• Training: Data à Models
• Predic9on: Models à Forecast
• Decision: Forecast à Ac9ons

ML: Robotics

ML: Robotics

ML: Image Recognition

Supervised Learning

Supervised Learning
Input Outcome

Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome

Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
Supervised
Learning
known historical data

Supervised Learning
Input Outcome
Input
Input
Input
Outcome
Outcome
Outcome
Supervised
Learning
Unseen Input Same Outcome
known historical data

Amazon Machine Learning Service

Amazon Machine Learning Service

Amazon Machine Learning Service

Amazon Machine Learning Service

Telco Churn Dataset
• US telco customers, their cell phone plans and usage
• 21 attributes, 3333 rows:
• Customer: State, Area_Code, Phone
• Plan: Intl_Plan, VMail_Plan
• Behavior: VMail_Messages, Day_Mins, Day_Calls,
Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,
Night_Mins, Night_Calls, Night_Charge, Intl_Mins,
Intl_Calls, Intl_Charge
• Other: Account_Length, CustServ_Calls, Churn

Telco Churn Dataset
• US telco customers, their cell phone plans and usage
• 21 attributes, 3333 rows:
• Customer: State, Area_Code, Phone
• Plan: Intl_Plan, VMail_Plan
• Behavior: VMail_Messages, Day_Mins, Day_Calls,
Day_Charge, Eve_Mins, Eve_Calls, Eve_Charge,
Night_Mins, Night_Calls, Night_Charge, Intl_Mins,
Intl_Calls, Intl_Charge
• Other: Account_Length, CustServ_Calls, Churn

Telco Churn Dataset
KS, 128, 415, 382-4657, 0, 1, 25, 265.100000, 110, 45.070000, 197.400000, 99,
16.780000, 244.700000, 91, 11.010000, 10.000000, 3, 2.700000, 1, 0
OH, 107, 415, 371-7191, 0, 1, 26, 161.600000, 123, 27.470000, 195.500000, 103,
16.620000, 254.400000, 103, 11.450000, 13.700000, 3, 3.700000, 1, 0
NJ, 137, 415, 358-1921, 0, 0, 0, 243.400000, 114, 41.380000, 121.200000, 110,
10.300000, 162.600000, 104, 7.320000, 12.200000, 5, 3.290000, 0, 0
OH, 84, 408, 375-9999, 1, 0, 0, 299.400000, 71, 50.900000, 61.900000, 88, 5.260000,
196.900000, 89, 8.860000, 6.600000, 7, 1.780000, 2, 0
OK, 75, 415, 330-6626, 1, 0, 0, 166.700000, 113, 28.340000, 148.300000, 122, 12.610000,
186.900000, 121, 8.410000, 10.100000, 3, 2.730000, 3, 0
AL, 118, 510, 391-8027, 1, 0, 0, 223.400000, 98, 37.980000, 220.600000, 101, 18.750000,
203.900000, 118, 9.180000, 6.300000, 6, 1.700000, 0, 0

Creating Datasource for Amazon ML

Creating Datasource for Amazon ML

Building the Amazon ML Model

Recipe
{ "groups": {
"NUMERIC_VARS_NORM":
"group('Intl_Charge','Night_Calls','Day_Calls','Eve_Calls','Eve_Mins','Int
l_Mins','VMail_Message','Intl_Calls','Day_Mins','Night_Mins','Day_Charge',
'Night_Charge','Eve_Charge','Account_Length')” },
"assignments": {},
"outputs": [
"ALL_BINARY",
"State",
"Area_Code",
"normalize(NUMERIC_VARS_NORM)",
"CustServ_Calls"
]
}

Recipe: normalize() function
Account_Length Normalized Value
128 0.808771865
107 -0.047574816
137 1.175777586
84 -0.985478323
75 -1.352484044
118 0.400987732

Building the Amazon ML Model

Cost of Errors
• Cost of Customer Churn and Acquisition (false negative):
• foregone cashflow
• advertising costs
• POS and sign-up admin costs
• Customer Retention Cost (false + true positive)
• Discounts
• Phone upgrades
• etc

Financial Outcome of Applying a Model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
False Negative True + False Pos Retention Cost Cost with ML
4.80% 26.40% $100.00 $50.40

Financial Outcome of Applying a Model
Prior Churn Churn Cost Cost without ML
14.49% $500.00 $72.46
False Negative True + False Pos Retention Cost Cost with ML
4.80% 26.40% $100.00 $50.40
• $22.06 of savings per customer
• With 100,000 customers over $2MM in savings with ML

@dbatalov