Presented an abridged version of my "What is data science" talk at #websummit 2013. This talk goes over the required skillset as defined by Drew Conway and his famous venn diagram, and also outlines the Data Scientific Method brought by Dr. Patil. The talk is mainly two parts and the second part goes over some of the packages and technologies we use — minus the storage part.

1. Data Science?!
what even...

2. David Coallier
@davidcoallier

3. Data Scientist
Engine Yard

5. And I cook..
A lot.

6. (n-1) items

7. Adapting.

8. Feedback.

9. Indifference.

10. Young mathematically
inclined minds

11. Young mathematically inclined minds
We knew everything.

12. First Bad Assumption.

13. So we asked “experts”.

14. Wrong Ingredients

15. Bad Data

16. Tasted like sh*t

20. From Our Results
We had questions.

21. Found Expertise
Not Online.

22. Data Scientific
Method

23. Find a Question
Your Hypothesis

24. Current Data
What do you have?

25. Features & Tests
Try it.

26. Analyse Results
Won’t be pretty.

27. Conversation
Framed. By. Data.

28. But....

29. Good Discussions
Imply good data scientists

30. Hacking Skills

31. Hacking Skills
Maths &
Stats

32. Hacking Skills
Expertise
Maths &
Stats

33. Hacking Skills
Machine
Learning
Danger
Zone!!!
Expertise
Research
Maths &
Stats

34. Hacking Skills
Data
Science
Expertise
Maths &
Stats

35. Hacking Skills
Danger
Zone!!!
Machine
Learning
Data
Science
Maths &
Stats
Expertise
Research

36. Business
Don’t need an MBA

37. In other words.

38. 1. Hacking
2. Maths & Stats
3. Expertise

39. Apply Method
Data Scientific

40. 1. Question
2. Current Data
3. Features/Tests
4. Analyse
5. Converse

41. Find a Question
Let’s imagine Github

42. Upgrade Repos
Affect users as little as possible

43. import csv
content = csv.read('repo1.csv')

44. λ e
f (k; λ ) =
k!
k −k
for k >= 0

46. Converse
Present Findings

47. Iterate
Commits aren’t key.

48. KPIs are key
Indicators from experience

49. Questions
Super Important.

50. Just test it..

54. We are Human.
Emotional Connection

55. What next?
Second Hypothesis.

57. Focus on Data
Relevant to your KPIs.

59. Data gives you the what
Humans give you the why

60. Turn Information

61. Into
Actionable Insight

62. Create Discussions
Introspection Engines

63. Seeing, Feeling it
The brain sees.

64. Not regressions

65. Not p-values

66. Not slopes

67. Not F-statistics

68. Not coefficients

71. Question Data
Not Visualisations.

72. Toolbox
What do we use?

73. R
Modeling, Testing, Prototyping

74. RStudio
The IDE

75. lubridate
and zoo
Dealing with Dates...

76. yy/mm/dd
mm/dd/yy
YYYY-mm-dd HH:MM:ss TZ
yy-mm-dd
1363784094.513425
yy/mm
different timezone

77. reshape2
Reshape your Data

78. ggplot2
Visualise your Data

79. RCurl, RJSONIO
Find more Data

80. HMisc
Miscellaneous useful functions

81. forecast
Can you guess?

82. garch
Generalized Autoregressive
Conditional Heteroskedasticity

83. quantmod
Statistical Financial Trading

85. getSymbols('AAPL')
barChart(AAPL)
addMACD()

86. xts
Extensible Time Series

87. igraph
Study Networks

88. maptools
Read & View Maps

89. map('state', region = c(row.names(USArrests)), col=cm.colors(16, 1)[floor(USArrests$Rape/max(USArrests$Rape)*28)], fill=T)

90. Python
Scientific Computing

91. SciPy
http://www.scipy.org

92. scipy.stats

93. scipy.stats
Descriptive Statistics

94. from scipy.stats import
describe
s = [1,2,1,3,4,5]
print describe(s)

95. scipy.stats
Probability Distributions

96. Example
Poisson Distribution

97. λ e
f (k; λ ) =
k!
k −k
for k >= 0

98. import scipy.stats.poisson
p = poisson.pmf([1,2,3,4,1,2,3], 2)

99. print p.mean()
print p.sum()
...

100. NumPy
http://www.numpy.org/

101. NumPy
Linear Algebra

102. ⎛ 1 0 ⎞
⎜ 0 1 ⎟
⎝
⎠

103. import numpy as np
x = np.array([ [1, 0], [0, 1] ])
vec, val = np.linalg.eig(x)
np.linalg.eigvals(x)

104. >>> np.linalg.eig(x)
(
array([ 1., 1.]),
array([
[ 1., 0.],
[ 0., 1.]
])
)

105. Matplotlib
Python Plotting

106. statsmodels
Advanced Statistics Modeling

107. NLTK
Natural Language Tool Kit

108. scikit-learn
Machine Learning

109. from sklearn import tree
X = [[0, 0], [1, 1]]
Y = [0, 1]
clf = tree.DecisionTreeClassifier()
clf = clf.fit(X, Y)
clf.predict([[2., 2.]])
>>> array([1])

110. PyBrain
... Machine Learning

111. PyMC
Bayesian Inference

112. Pattern
Web Mining for Python

113. NetworkX
Study Networks

114. MILK: Machine Learning

115. Pandas
easy-to-use data structures

116. from pandas import *
x = DataFrame([
{"age": 26},
{"age": 19},
{"age": 21},
{"age": 18}
])
print x[x['age'] > 20].count()
print x[x['age'] > 20].mean()

117. Python vs R?
Different Purposes

118. Dogfooding
Data Scientific Method

119. Original Question
What is Data Science?

120. Back to you
For questioning