"Data Science at The New York Times" talk given to the students of IACS, Harvard 2014-10-31 announcement: http://www.seas.harvard.edu/calendar/event/77966 article: http://www.thecrimson.com/article/2014/11/3/data-scientist-times-seminar/

1. data science @ The New York Times
!
(and what can academia do for a
163-year old company?)
chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
@chrishwiggins

2. 0. the path

3. biology: 1892 vs. 1995
biology changed for good.

4. genetics: 1837 vs. 2012
from “segments” to algorithms

5. genetics: 1837 vs. 2012
from intuition to prediction

6. genetics: 1837 vs. 2012
ML toolset; data science mindset

7. data science: web scale

8. example:
163 yr old

9. bit.ly/nyt-interactive-2013

10. R+D: nytlabs.com

14. example:
millions of views per hour

16. data science: the web

19. learning the “genome” of loyal subscribers
insert figure here

20. from “segments” to algorithms
insert figure here

21. from intuition to prediction
insert figure here

22. data science: the web

23. data science: the web
is your “online presence”

24. data science: the web
is a microscope

25. data science: the web
is an experimental tool

26. data science: the web
is an optimization tool

27. 0. the path

28. 1. common tools

29. 1. common tools
- supervised learning
- unsupervised learning
- reinforcement learning

30. supervised learning, e.g.,

31. supervised learning, e.g.,
“the funnel”

32. interpretable supervised
learning
supercoolstuff

33. supervised learning, e.g.,
“logistics”

34. unsupervised learning, e.g,
“segments”

35. unsupervised learning, e.g,
“segments”

36. unsupervised learning, e.g,
“segments”
argmax_z p(z|x)=14

37. unsupervised learning, e.g,
“segments”
“soccer mom”

38. reinforcement learning

39. reinforcement learning
aka “A/B testing”;
RCT

40. reinforcement learning

41. reinforcement learning
img: MSR SV (RIP)
e.g., multi-armed bandits

42. 1. common tools
- supervised learning
- unsupervised learning
- reinforcement learning

43. 2. differing goals

44. “data”:

45. “data”:
“metrics”
“business analytics”
“Excel”
“reporting”

46. Reporting

47. Reportingbusiness as usual

48. Reporting
Learning
business as usual

49. Reporting
Learning(esp. supervised)
business as usual

50. Reporting
Learning
Test
business as usual
(esp. supervised)

51. Reporting
Learning
Test
aka “A/B testing”;
RCT
business as usual
(esp. supervised)

52. Reporting
Learning
Test
Optimizing
aka “A/B testing”;
RCT
(esp. supervised)
business as usual

53. Reporting
Learning
Test
Optimizing
aka “A/B testing”;
RCT
(i.e. reinforcement
learning)
(esp. supervised)
business as usual

54. Reporting
Learning
Test
Optimizing
Explore
aka “A/B testing”;
RCT
(i.e. reinforcement
learning)
(esp. supervised)
business as usual

55. Reporting
Learning
Test
Optimizing
Explore
aka “A/B testing”;
RCT
aka “segmenting”
(i.e. reinforcement
learning)
(esp. supervised)
business as usual

56. Reporting
Learning
Test
Optimizing
Explore

57. Reporting
Learning
Optimizing
tech co

58. Reporting
Optimizing
big co

59. Reporting
theoretical co

60. Reporting
Learning
Test
Optimizing
Explorestartups:

61. every publisher is now a startup

62. 3. creating a DS culture

63. DS team functions:
what does DS deliver?

64. DS team functions:
what does DS deliver?
- build data products
- build APIs
- impact roadmaps

65. - build data products

66. - build data products

67. - build data products
(including internal products)

68. - build APIs

69. - build APIs

70. - impact roadmaps
flickr/McJex

71. DS&E team composition
flickr/eggsalad78

72. DS&E team composition
- data engineering
- data science
- data visualization
- data product

73. 4. doing DS well

74. common requirements
in data science:

75. common requirements
in data science:
1. people
2. ideas
3. things
cf. USAF

76. data science: things

77. data science: things
- find quantifiables
!

78. data science: things
- find quantifiables (choose carefully)
!

79. data science: things
- straw man first
!

80. data science: things
- straw man first
!

81. data science: things
- small wins before feature engineering
!

82. data science: things
- data engineering before data science
!

83. data science: ideas

84. data science: ideas
- reframe domain questions
as machine learning tasks

85. data science: ideas
- better wrong than "nice"

86. data science: ideas
- be relevant
!

87. data science: ideas
- be relevant
!

88. data science: ideas
- be relevant
!

89. data science: ideas
- befriend experimentalists
!

90. data science: ideas
- befriend experimentalists
!

91. data science: ideas
- befriend experimentalists
!

92. data science: ideas
- befriend experimentalists
!
supercoolstuff

93. data science: people

94. data science: people
- be communicative
!

95. data science: people
- be communicative
(promote rhetorical literacy)

96. data science: people
- be communicative
(promote rhetorical literacy)
- related: strive to build models
which are both predictive and
interpretable

97. data science: people
- be skeptical
(promote critical literacy)

98. data science: people
- be empowering
!

99. data science: people
- be transparent
!

100. data science: people
- promote literacy:
functional
critical
rhetorical
!
(cf. Selber, Multiliteracies for a Digital Age. 2004)

101. data science: people
- promote literacies:
1. functional
2. critical
3. rhetorical
!
(cf. Selber, Multiliteracies for a Digital Age. 2004)

102. data science: people
- promote literacies:
1. functional
2. critical
3. rhetorical
!
(cf. Selber, Multiliteracies for a Digital Age. 2004)

103. data science: people
- promote literacies:
1. functional
2. critical
3. rhetorical
!
(cf. Selber, Multiliteracies for a Digital Age. 2004)

104. data science: people
- promote literacies:
1. functional
2. critical
3. rhetorical
!
(cf. Selber, Multiliteracies for a Digital Age. 2004)

105. summary:
pay attention to:
1. people
2. ideas
3. things
cf. USAF

106. people:
1. be communicative
2. be skeptical
3. be empowering
4. be transparent
5. promote literacies

107. ideas:
1. reframe questions as ML
2. better wrong than "nice"
3. be relevant
4. aim for hypothesis vs data jeapordy
5. befriend experimentalists

108. things:
1. find quantifiables
2. straw man first
3. small wins before feature engineering
4. data engineering before data science
!

109. find out more!
1. postdoc/student opportunities:
chris.wiggins@columbia.edu
!
2. we are hiring!
chris.wiggins@nytimes.com
!
3. let’s talk:
chris.wiggins@
@chrishwiggins

110. chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
@chrishwiggins