Big Data Analytics: The Math, the Implementation and How it can be Effectively Used to Reach Customers http://www.infotrustllc.com

1. UP NEXT…
10:00am
Big Data Analytics: The
Math, the Implementation
and How it can be
Effectively Used to Reach
Customers
Follow the action on Twitter using #AtE2014
BECK NADIR

2. Big Data Analytics:
The math, the implementation, and how it is
used to reach customers
By Beck Nadir
10/15/14 2!

3. 3!
By the Time You Walk Out of Here, You…
Will not be afraid of statistics!

4. 4!
By the Time You Walk Out of Here, You…
Will not be afraid of statistics!
Want to predict behavior

5. 5!
The Person Occupying Your Lunchtime!
How am I here?
- Huge nerd from the start
- Web analytics personnel at Moz

6. 6!
The Person Occupying Your Lunchtime!
How am I here?
- Huge nerd from the start
- Web analytics personnel at Moz
Education
- B.S. in Nuclear Engineering from Purdue University
- MBA from University of Washington – Bothell

7. 7!
Synopsis
Can every action in life be calculated?

8. We Will Go Over…
How do we track data, and why do we care?
8!

9. 9!
We Will Go Over…
What tools do we use to track and capture data?

10. 10!
We Will Go Over…
Mathematics!

11. 11!
We Will Go Over…
How do we make sense out of data?

12. 12!
We Will Go Over…
Can we predict future customer behavior at Moz?

13. 13!
We Will Go Over…
Customer value, and TAGFEE culture

14. 14!
How We Track Data, and the Tools We Use

15. 15!
How We Track Data, and the Tools We Use

16. 16!
Mathematics
The all watching eye?

17. 17!
Mathematics
What if I wanted to get personal…REALLY personal?

18. What if I wanted to get personal…REALLY personal?
“…When someone suddenly starts buying lots of
scent-free soap and extra-big bag of cotton balls, in
addition to hand sanitizers and washcloths, it signals
they could be getting close to their delivery date”.
(Target, 2012)
18!
Mathematics

19. “…because people who are going through a divorce
are more likely to miss payments,
your domestic troubles are of great interest to a
company that thrives on risk management.
Exactly how the credit industry does it (predict
divorce) – through sophisticated data-mining
techniques – is a closely guarded secret.” (Visa,
2010)
19!
Mathematics
What if I wanted to get personal…REALLY personal?

20. 20!
Mathematics
Would we do this at Moz?

21. 21!
Mathematics
Case Study:

22. 22!
Mathematics
Case Study:
Is there a way to expect a certain salary?

23. 23!
Mathematics
Case Study:
Is there a way to expect a certain salary?
Does education matter?

24. 24!
Mathematics
Case Study:
Is there a way to expect a certain salary?
Does education matter?
Advanced Degree?

25. 25!
Mathematics
Case Study:
Is there a way to expect a certain salary?
Does education matter?
Advanced Degree?
Gender?

26. 26!
Mathematics
What if…
Salary = 49708.65 + 424.78*yrs_exp +
1723.46*yrs_educ + 153.47*supervis +
1280.52*perform – 4372.53*female + 1239.95*MBA

27. 27!
Mathematics
What if…
…we know what to expect at all times?

28. 28!
Mathematics
All data we have, says something about the future.

29. 29!
Mathematics
All data we have, says something about the future.
It’s a question of probability, and independent variables.

30. 30!
Mathematics
All data we have, says something about the future.
It’s a question of probability, and independent variables.
Bond, James Bond!

31. 31!
Mathematics
http://www.youtube.com/watch?v=l5C7LMOWyYc

32. 32!
Mathematics
We always start with lots of data

33. 33!
Mathematics
We always start with lots of data
Nm = Number of males= 24. Nf = Number of females = 17.
Xm = Average male management salary = $68,609.
Xf = Average female management salary = $65,763.
Sm = Male salary standard deviation = $6,108.
Sf = Female salary standard deviation = $6,084

34. Mathematics
We always start with lots of data
The trick is making sense out of it
Nm = Number of males= 24. Nf = Number of females = 17.
Xm = Average male management salary = $68,609.
Xf = Average female management salary = $65,763.
Sm = Male salary standard deviation = $6,108.
Sf = Female salary standard deviation = $6,084
34!

35. 35!
Mathematics
Overwhelming!

36. 36!
Mathematics
3 Common Ways of Creating Predictive Analytics:

37. 37!
Mathematics
3 Common Ways of Creating Predictive Analytics:
a. Multi-Colinearity Analysis
§ Practice of finding and relating one variable (KPI) to
another.
§ The less related two variables are to each other, the
better for analysis.

38. 38!
Mathematics
3 Common Ways of Creating Predictive Analytics:
a. Multi-Colinearity Analysis
Compare/Contrast

39. 39!
Mathematics
3 Common Ways of Creating Predictive Analytics:
b. Linear/Multi-Linear Regression
§ Where statistics come together, to predict a future event.
§ A series of variables, determines a single outcome.

40. 40!
Mathematics
3 Common Ways of Creating Predictive Analytics:
b. Linear/Multi-Linear Regression
Fit the Pieces

41. 41!
Mathematics
3 Common Ways of Creating Predictive Analytics:
c. Cluster Analysis
§ Practice of grouping data points in similar “clusters”.
§ Practice of statistical distribution, and multi-objective
optimization.

42. 42!
Mathematics
3 Common Ways of Creating Predictive Analytics:
c. Cluster Analysis
Group the Knowledge

43. 43!
Mathematics
3 Common Ways of Creating Predictive Analytics:

44. Mathematics
3 Common Ways of Creating Predictive Analytics:
Compare/Contrast
44!

45. 3 Common Ways of Creating Predictive Analytics:
Compare/Contrast
45!
Mathematics
Fit the Pieces

46. 3 Common Ways of Creating Predictive Analytics:
Compare/Contrast
46!
Mathematics
Fit the Pieces Group the Knowledge

47. 47!
How Do We Make Sense Out of Data?
Does Gender and/or education effect salary?

48. 48!
How Do We Make Sense Out of Data?
Does Gender and/or education effect salary?
Case Study:
Harvard Review’s Equal Pay for Equal Work

49. 49!
How Do We Make Sense Out of Data?
Multi-Colinearity Analysis

50. 50!
How Do We Make Sense Out of Data?
Multi-Colinearity Analysis
Compare/Contrast

51. 51!
How Do We Make Sense Out of Data?
Multi-Colinearity Analysis
Gender y = -9E-06x + 1.0371
R² = 0.02627
45000 50000 55000 60000 65000 70000 75000 80000 85000 90000 Salary ($)
1
0.5
0
Gender vs. Salary

52. 52!
How Do We Make Sense Out of Data?
Multi-Colinearity Analysis
Years
Exp.
Yrs.
Educa.on
Supervisor
Exp.
Performance
Female
MBA
Salary
Salary
Last
Job
How
Much
Asked
For
Ambi.on
Salary
0.58
0.55
0.58
0.59
0.52
0.57
1
0.85
0.9
0.75

53. 53!
How Do We Make Sense Out of Data?
Multi-Colinearity Analysis
Gender vs. Salary
y = -9E-06x + 1.0371
R² = 0.02627
1
0.5
0
45000 55000 65000 75000 85000
Gender
Salary ($)
Years
Exp.
Yrs.
Educa.on
Supervisor
Exp.
Performance
Female
MBA
Salary
Salary
Last
Job
How
Much
Asked
For
Ambi.on
Salary
0.58
0.55
0.58
0.59
0.52
0.57
1
0.85
0.9
0.75

54. 54!
How Do We Make Sense Out of Data?
Transition from Multi-Colinearity Analysis…

55. 55!
How Do We Make Sense Out of Data?
Transition from Multi-Colinearity Analysis…
Now that we see a correlation, does this
mean causation?

56. How Do We Make Sense Out of Data?
Linear Regression
56!

57. How Do We Make Sense Out of Data?
Linear Regression
57!
Fit the Pieces

58. How Do We Make Sense Out of Data?
Linear Regression
58!
Ask a basic question:

59. How Do We Make Sense Out of Data?
Linear Regression
59!
Ask a basic question:
Does Gender and/or education effect salary?

60. How Do We Make Sense Out of Data?
Linear Regression
60!
Ask a basic question:
Null Hypothesis = Ho = Mm – Mf = 0

61. How Do We Make Sense Out of Data?
Linear Regression
61!
Ask a basic question:
Null Hypothesis = Ho = Mm – Mf = 0
Alternate Hypothesis = Ha = Mm – Mf > 0 (Men make higher
salaries).

62. How Do We Make Sense Out of Data?
Linear Regression
Determine T_critical – The maximum threshold disproving Ho.
62!

63. How Do We Make Sense Out of Data?
Linear Regression
Determine T_critical – The maximum threshold disproving Ho.
df (Degrees of Freedom) = N – 2 = 41 – 2 = 39
63!

64. How Do We Make Sense Out of Data?
Linear Regression
Determine T_critical – The maximum threshold disproving Ho.
df (Degrees of Freedom) = N – 2 = 41 – 2 = 39
To decide:
α = Alpha = Margin of error = 5% (95% certainty)
64!

65. How Do We Make Sense Out of Data?
Linear Regression
65!

66. How Do We Make Sense Out of Data?
Linear Regression
66!
T_critical = 1.7

67. How Do We Make Sense Out of Data?
Linear Regression
Calculate T_actual – difference between the two means
67!

68. How Do We Make Sense Out of Data?
Linear Regression
Calculate T_actual – difference between the two means
68!
T _ actual =
(Xm− Xf )−(Mm−Mf )
Sm2 (Nm−1)+ Sf 2 (Nf −1)
Nm+ Nf − 2
1
Nm
+
1
Nf

69. How Do We Make Sense Out of Data?
Linear Regression
Calculate T_actual – difference between the two means
69!
T_actual = 1.47
T _ actual =
(Xm− Xf )−(Mm−Mf )
Sm2 (Nm−1)+ Sf 2 (Nf −1)
Nm+ Nf − 2
1
Nm
+
1
Nf

70. How Do We Make Sense Out of Data?
Linear Regression
If Tactual is < Tcritical, reject the Null Hypothesis.
70!

71. How Do We Make Sense Out of Data?
Linear Regression
If Tactual is < Tcritical, reject the Null Hypothesis.
71!
1.47 < 1.7

72. How Do We Make Sense Out of Data?
Linear Regression
If Tactual is < Tcritical, reject the Null Hypothesis.
72!
1.47 < 1.7
In this case, evidence shows women in management make
less than male counterparts.

73. How Do We Make Sense Out of Data?
Linear Regression
73!
Is there a margin of error?

74. How Do We Make Sense Out of Data?
Linear Regression
74!
Is there a margin of error?
Confidence Interval Test:
(Xm− Xf )−(tα /2 *γ ) ≤μ1 −μ2 ≤ (Xm− Xf )+(tα /2 *γ )

75. How Do We Make Sense Out of Data?
75!
Confidence Interval Test:
(Xm− Xf )−(tα /2 *γ ) ≤μ1 −μ2 ≤ (Xm− Xf )+(tα /2 *γ )
Where:
γ =
Sm2 (Nm−1)+ Sf 2 (Nf −1)
Nm+ Nf − 2
1
Nm
+
1
Nf
Linear Regression
Is there a margin of error?

76. How Do We Make Sense Out of Data?
Linear Regression
76!
Is there a margin of error?
γ = 1933 and Tα/2 = 2.0

77. How Do We Make Sense Out of Data?
Linear Regression
77!
Is there a margin of error?
γ = 1933 and Tα/2 = 2.0
Therefore:
-$1,020 ≤ Mm – Mf ≤ $6,712

78. How Do We Make Sense Out of Data?
Linear Regression
78!
Is there a margin of error?

79. How Do We Make Sense Out of Data?
Linear Regression
79!
Is there a margin of error?
Men could be making anywhere between $1,020 less,
or $6,712 more than women.

80. How Do We Make Sense Out of Data?
Linear Regression
80!
What if I wanted to know more…what else affects pay?

81. How Do We Make Sense Out of Data?
Linear Regression
81!

82. How Do We Make Sense Out of Data?
Linear Regression
What if I wanted to dig even more…do education and MBA
affect pay?
82!

83. How Do We Make Sense Out of Data?
Linear Regression
83!

84. How Do We Make Sense Out of Data?
Linear Regression
84!
What if I wanted to specify groups to target?

85. How Do We Make Sense Out of Data?
Linear Regression
85!
What if I wanted to specify groups to target?
Don’t worry, we can use math for that too!

86. How Do We Make Sense Out of Data?
Transition from Linear Regression…
86!

87. How Do We Make Sense Out of Data?
Transition from Linear Regression…
We have lots of equations and linear
regressions. What do we do with them?
87!

88. How Do We Make Sense Out of Data?
Cluster Analysis
88!

89. How Do We Make Sense Out of Data?
89!
Group the Knowledge
Cluster Analysis

90. How Do We Make Sense Out of Data?
Cluster Analysis
90!
What kind of story can I tell about these clusters?

91. How Do We Make Sense Out of Data?
Cluster Analysis
91!
What kind of story can I tell about these clusters?
Discriminant
Variable
Cluster
1
Gender
1.406
Age
20-­‐35
Educa.on
(Years)
16-­‐20
HH
Income
$40,000
-­‐
$65,000
No.
of
Children
0-­‐2
Conserva.ve?
0.275
Liberal?
0.801
Fun
Loving?
0.622
Cung
Edge?
0.717
Family
Oriented?
0.087
Trendy?
0.645

92. How Do We Make Sense Out of Data?
Cluster Analysis
92!
What kind of story can I tell about these clusters?
Discriminant
Variable
Cluster
2
Gender
1.404
Age
36-­‐45
Educa.on
(Years)
16-­‐20
HH
Income
$66,000
-­‐
$90,000
No.
of
Children
2-­‐4
Conserva.ve?
0.769
Liberal?
0.322
Fun
Loving?
0.565
Cung
Edge?
0.529
Family
Oriented?
0.553
Trendy?
0.594

93. How Do We Make Sense Out of Data?
Cluster Analysis
93!
What kind of story can I tell about these clusters?
Discriminant
Variable
Cluster
3
Gender
1.102
Age
46-­‐60
Educa.on
(Years)
12
HH
Income
$40,000
-­‐
$65,000
No.
of
Children
4-­‐6
Conserva.ve?
0.822
Liberal?
0.294
Fun
Loving?
0.443
Cung
Edge?
0.327
Family
Oriented?
0.822
Trendy?
0.293

94. How Do We Make Sense Out of Data?
Cluster Analysis
94!
What kind of story can I tell about these clusters?
Left brain…Meet the right brain!

95. How Do We Make Sense Out of Data?
95!
Not all predictions will be correct.

96. How Do We Make Sense Out of Data?
96!
Not all predictions will be correct.
“The Denver Broncos defeated the Seattle Seahawks
31-28 in the official EA SPORTS prediction of Super
Bowl XLVIII”.

97. 97!
Can We Predict Future Behavior at Moz?

98. 98!
Can We Predict Future Behavior at Moz?
How can we better help our customers?

99. 99!
Can We Predict Future Behavior at Moz?
How can we better help our customers?
Signs of churn?

100. Can We Predict Future Behavior at Moz?
100!

101. Can We Predict Future Behavior at Moz?
101!

102. Can We Predict Future Behavior at Moz?
Matt Peters, Moz Data Scientist
Alyson Murphy, Senior Data Analyst
Nick Sayers, Dir. of Customer Success and
Support
102!

103. Can We Predict Future Behavior at Moz?
Proactively engaging Free Trialers
(via chat and e-mail).
103!

104. Can We Predict Future Behavior at Moz?
Proactively engaging Free Trialers
(via chat and e-mail).
Increasing vesting rate by 6.63%!
104!

105. Can We Predict Future Behavior at Moz?
Proactively engaging Free Trialers
(via chat and e-mail).
Increasing vesting rate by 6.63%!
What actions or activities should we
encourage customers to do?
105!

106. Can We Predict Future Behavior at Moz?
106!
Free Trials longer than 1 month vest at a
lower rate

107. Can We Predict Future Behavior at Moz?
107!
Prior Pro members vest at nearly TWICE
the rate as first time customers. We should
streamline their re-entry process.

108. Can We Predict Future Behavior at Moz?
108!
Community members vest at a higher rate.

109. Can We Predict Future Behavior at Moz?
109!
Users are most engaged
during the first few days of
their trial.

110. 110!
Can We Predict Future Behavior at Moz?
Usage of MA and OSE
drops to less then a few
clicks / user after 10 days.

111. 111!
Can We Predict Future Behavior at Moz?
Most campaigns are created
during the first two days.

112. 112!
Can We Predict Future Behavior at Moz?
Setting up a campaign is essential to vesting rate.

113. 113!
Can We Predict Future Behavior at Moz?
What else should we look at?

114. 114!
Customer Value and TAGFEE Culture

115. 115!
Customer Value and TAGFEE Culture
We should also make sure:

116. 116!
Customer Value and TAGFEE Culture
We should also make sure:
- The customer experience is personalized.

117. 117!
Customer Value and TAGFEE Culture
We should also make sure:
- The customer experience is personalized.
- Realize not everyone will want to be chatted!

118. 118!
Customer Value and TAGFEE Culture
We should also make sure:
- The customer experience is personalized.
- Realize not everyone will want to be chatted!
- Customers realize the full value of Moz.

119. 119!
Customer Value and TAGFEE Culture

120. Customer Value and TAGFEE Culture
Transparent
120!

121. Customer Value and TAGFEE Culture
Transparent
Authentic
121!

122. Customer Value and TAGFEE Culture
Transparent
Authentic
Generous
122!

123. Customer Value and TAGFEE Culture
Transparent
Authentic
Generous
Fun
123!

124. Customer Value and TAGFEE Culture
Transparent
Authentic
Generous
Fun
Empathetic
124!

125. Customer Value and TAGFEE Culture
Transparent
Authentic
Generous
Fun
Empathetic
Exceptional
125!

126. Conclusion:
126!

127. Conclusion:
127!

128. Conclusion:
Compare/Contrast
128!

129. Conclusion:
Compare/Contrast Fit the Pieces
129!

130. Conclusion:
Compare/Contrast Fit the Pieces Group the Knowledge
130!

131. Conclusion:
Subscribers are not just another customer!
131!

132. Conclusion:
Subscribers are not just another customer!
Our help team answers all questions, one by one!
132!

133. Conclusion:
Subscribers are not just another customer!
Our help team answers all questions, one by one!
Founder, and CEO, interact with subscribers regularly
133!

134. Conclusion:
Subscribers are not just another customer!
Our help team answers all questions, one by one!
Founder, and CEO, interact with subscribers regularly
Moz Community!
134!

135. Conclusion:
Subscribers are not just another customer!
Our help team answers all questions, one by one!
Founder, and CEO, interact with subscribers regularly
Moz Community!
135!

136. Homework!
Dig through your data!
136!

137. Homework!
Dig through your data!
Are there metrics you can relate to each other?
137!

138. Homework!
Dig through your data!
Are there metrics you can relate to each other?
What factors make up revenue (or a key metric) in
your businesses?...hypothesis test, fit them together!
138!

139. Homework!
Dig through your data!
Are there metrics you can relate to each other?
What factors make up revenue (or a key metric) in
your businesses?...hypothesis test, fit them together!
Have you segmented your customers? What groups
do they represent?
139!

140. Homework!
Dig through your data!
Are there metrics you can tie revenue to?
What factors make up revenue in your
businesses?...fit them together!
Have you segmented your customers? What groups
do they represent?
Make mistakes!
140!

141. Thanks for Watching!
LinkedIn: Beck Nadir
Twitter: @annalesparrales 141!

142. References
Calculator picture, page 6
http://pixabay.com/en/calculator-calculation-insurance-385506/
Caring picture, page 7
http://pixabay.com/en/care-feeling-female-couple-give-20185/
Ciarelli, Nicholas (2010, April 6). How Visa Predicts Divorce. Retrieved March 24, 2013, from: www.dailybeast.com.
142!
http://www.thedailybeast.com/articles/2010/04/06/how-mastercard-predicts-divorce.html
Denver Broncos prediction, page 95:
http://www.easports.com/madden-nfl/news/2014/super-bowl-48-prediction
Hill, Kashmir (2012, February 16). How Target Figured Out a Teen Girl was Pregnant Before Her Father Did. Retrieved March
25, 2013, from: www.forbes.com.
http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/
Adobe Omniture Site Catalyst Example Provided by:
https://help.optimizely.com/hc/en-us/articles/200039985-Integrating-Optimizely-with-Adobe-Analytics-Omniture-
SiteCatalyst-
Retail Dashboard Example Provided by:
http://www.dashboardinsight.com/dashboards/live-dashboards/financial-operations-dashboard-dundas.aspx

143. References
Compare contrast picture, pages 37, 43, 44, 45, 49, 127, 128, 129
http://en.wikipedia.org/wiki/Apples_and_oranges
Database picture, page 15
http://pixabay.com/en/database-data-storage-cylinder-149760/
Hadoop logo, page 13,125
http://commons.wikimedia.org/wiki/File:Apache_Hadoop_Elephant.jpg
SQL Server: 13, 125
http://commons.wikimedia.org/wiki/File:Sql-server-ce-4-logo.png
Tetris picture, page 39, 44, 45, 56, 128, 129
http://commons.wikimedia.org/wiki/File:Tetrominoes_IJLO_STZ_Worlds.svg
Nerd picture, page 4, 5
http://pixabay.com/en/nerd-scientist-chemist-physicist-155841/
Tool picture, page 8
http://pixabay.com/en/tool-pliers-screwdriver-145375/
Math formula picture, page 9
http://pixabay.com/en/math-function-symbol-icon-27248/
143!

144. References
Question Mark Picture, page 10, 11
http://commons.wikimedia.org/wiki/File:Red_question_mark.png
Multiple Question Marks, page 26-29
http://pixabay.com/fr/point-d-interrogation-questions-63979/
James Bond Scene, page 30
http://www.youtube.com/watch?v=l5C7LMOWyYc
Overwhelmed Picture, page 34
http://pixabay.com/en/mimic-panic-scratch-woman-person-156928/
Group Picture, page 41, 45, 88, 129
http://pixabay.com/en/queue-communal-community-group-154925/
Crystal Ball Picture, page 101, 102, 103, 104
http://pixabay.com/en/crystal-ball-glass-globe-glass-ball-32381/
Dashboard Example Picture, page 14, 126
http://commons.wikimedia.org/wiki/File:Well_Organized_Dashboard_Example.jpg
Omniture Logo Picture, page 14, 126
http://commons.wikimedia.org/wiki/File:Omniture.png
144!