DataEngConf: Data Science at the New York Times by Chris Wiggins

•

5 gefällt mir•944 views

This document discusses data science at The New York Times. It references various topics related to data science including predictive analytics, descriptive analytics, prescriptive analytics, data engineering, data science skills, and the importance of people, ideas, and tools/delivery in data science teams. It also references the data science work of Chris Wiggins and how data science has evolved the field of journalism and publishing.

Software

data science @ The New York Times
chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
@chrishwiggins
references: bit.ly/icerm

data science: mindset & toolset
drew conway, 2010
references: bit.ly/icerm

modern history:
2009
references: bit.ly/icerm

biology: 1892 vs. 1995
references: bit.ly/icerm

biology: 1892 vs. 1995
references: bit.ly/icerm
new toolset, new mindset

news: 20th century
church state
references: bit.ly/icerm

news: 21st century
church state
engineering
references: bit.ly/icerm

news: 21st century
church state
engineering: DSE
references: bit.ly/icerm

1851 1996
newspapering: 1851 vs. 1996
references: bit.ly/icerm

references: bit.ly/icerm
"...social activities generate large quantities of potentially
valuable data...The data were not generated for the
purpose of learning; however, the potential for learning
is great’’

data science: the web
references: bit.ly/icerm

data science: the web
is your “online presence”
references: bit.ly/icerm

data science: the web
is a microscope
references: bit.ly/icerm

data science: the web
is an experimental tool
references: bit.ly/icerm

data science: the web
is an optimization tool
references: bit.ly/icerm

1851 1996
newspapering: 1851 vs. 1996 vs. 2008
2008
references: bit.ly/icerm

“a startup is a temporary organization in search of a
repeatable and scalable business model” —Steve Blank
references: bit.ly/icerm

every publisher is now a startup
references: bit.ly/icerm

learnings
- predictive analytics
- descriptive analytics
- prescriptive analytics
references: bit.ly/icerm

learnings
- predictive analytics
- descriptive analytics
- prescriptive analytics
cf. modelingsocialdata.org
references: bit.ly/icerm

predictive analytics, e.g.,
cf. modelingsocialdata.org

predictive analytics, e.g.,
“the funnel”
cf. modelingsocialdata.org

interpretable predictive analytics
supercoolstuff
cf. modelingsocialdata.org

interpretable predictive analytics
supercoolstuff
cf. modelingsocialdata.org
arxiv.org/abs/q-bio/0701021

optimization & learning, e.g.,
“How The New York Times Works “popular mechanics, 2015

optimization & prediction, e.g.,
“How The New York Times Works “popular mechanics, 2015
(some models)
(somemoneys)

recommendation as predictive analytics
bit.ly/AlexCTM

descriptive analytics, e.g,
cf. daeilkim.com ; import bnpy

Reporting
Learning
Test
Optimizing
Exploredescriptive:
predictive:
prescriptive:

common requirements in
data science:
1. people
2. ideas
3. things
cf. USAF

things:
what does DS team deliver?
- build data prototypes
- build APIs
- impact roadmaps

- build data prototypes
cf. daeilkim.com

- in puppet, w/python2.7
- collaboration w/pers. team
- build APIs

data skills
- data engineering
- data science
- data visualization
- data product
- data multiliteracies
- data embeds
cf. “data scientists at work”, ch 1

data science: people
- new mindset > new toolset

summary:
pay attention to:
1. people
2. ideas
3. things
cf. USAF

data science @ The New York Times
chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
@chrishwiggins

Weitere ähnliche Inhalte

Was ist angesagt?

Storytelling in the database era: uncertainty and science reporting

Paul Bradshaw

Data journalism

hamid obaidi

But Who Protects the Moderators?

Matthew Lease

Maps and data esri health care 2012

J T "Tom" Johnson

The s+a3 project: leveraging analytic resources

J T "Tom" Johnson

Big Data and the Social Sciences Ex-Google engineer Abe Usher presents a talk about Big Data technology and methods applicable to social science. Participants will learn techniques that are used by Google engineers to collect, clean, analyze, and visualize Big Data. Additionally Mr. Usher will provide URLs to sample data, open source applications, and code to those interested in applying these Big Data methods themselves.

Big Data and the Social Sciences

Abe Usher

Teaching AI in data journalism

Paul Bradshaw

Statistics in Journalism Sheffield 2014

Elpidio del Campo Cañizares

It's the people's data

J T "Tom" Johnson

Using Data for Science Journalism

Jonathan Gray

Dharmendra Rama

Computer assisted research and reporting

peterverweij

Data journalism presentation

Kwami Ahiabenu,II

It’s the people’s data presentation april 2015

J T "Tom" Johnson

Building Data-centric Media Organizations

J T "Tom" Johnson

10 ways AI can be used for investigations

Paul Bradshaw

Data-driven journalism: What is there to learn? (Stanford, June 2010) #ddj

Mirko Lorenz

Privately Obscure

lisafilipek

In an era of digitalization of data processes, the management accounting profession has the potential to count even more. They interpret big data and exercise judgement. This presentation will explore new and emerging information technology trends that can help the management accounting profession take a leadership role within their organization by exploring the interface of strategy, emerging information technologies, and of cost information. We will further explore how the management accountant can apply emerging information technology to maximise the value of information while minimising the costs and risks of holding it.

The Management Accountant in a Digital World The interface of strategy, tech...

Workiva

The Other HPC: High Productivity Computing

University of Washington

Was ist angesagt? (20)

Storytelling in the database era: uncertainty and science reporting

Data journalism

But Who Protects the Moderators?

Maps and data esri health care 2012

The s+a3 project: leveraging analytic resources

Big Data and the Social Sciences

Teaching AI in data journalism

Statistics in Journalism Sheffield 2014

It's the people's data

Using Data for Science Journalism

Dharmendra Rama

Computer assisted research and reporting

Data journalism presentation

It’s the people’s data presentation april 2015

Building Data-centric Media Organizations

10 ways AI can be used for investigations

Data-driven journalism: What is there to learn? (Stanford, June 2010) #ddj

Privately Obscure

The Management Accountant in a Digital World The interface of strategy, tech...

The Other HPC: High Productivity Computing

Ähnlich wie DataEngConf: Data Science at the New York Times by Chris Wiggins

850 keynote siegel

Rising Media, Inc.

Big data is one of the most popular terms in the IT industry during the past decade. The word is vague and broad enough that essentially every one of us is living in a big-data world. Every time you do a google search, like a post in Facebook, write something in WeChat or view some item on Amazon, you both use and contribute to someone's big data system. Managing so much data across many computers introduce unique challenges. In this talk, we review the landscape of big data platforms and discuss some lessons we learned from building them.

Big Data Platform Landscape by 2017

Donghui Zhang

Elsevier/Maryland Publishing Connect - 14_0331 (pdf)

jeffreylancaster

[데이터야 놀자 2018] 뉴욕에서 만난 데이터 사이언스

BigRadar

Getting to Know Your Data with R

Stephen Withington

Event Sourcing 101 (Vienna, 21. 2. 2019)

Jan Kuchař

Data Anayltics: How to predict anything

CONTACT Software

Roger hoerl say award presentation 2013

Roger Hoerl

Why, When, and How You Should Update Your Content (Raffaele Asquer, SearchLov...

Raffaele Asquer

A New Year in Data Science: ML Unpaused

Paco Nathan

Ähnlich wie DataEngConf: Data Science at the New York Times by Chris Wiggins (10)

850 keynote siegel

Big Data Platform Landscape by 2017

Elsevier/Maryland Publishing Connect - 14_0331 (pdf)

[데이터야 놀자 2018] 뉴욕에서 만난 데이터 사이언스

Getting to Know Your Data with R

Event Sourcing 101 (Vienna, 21. 2. 2019)

Data Anayltics: How to predict anything

Roger hoerl say award presentation 2013

Why, When, and How You Should Update Your Content (Raffaele Asquer, SearchLov...

A New Year in Data Science: ML Unpaused

Mehr von Hakka Labs

Always Valid Inference (Ramesh Johari, Stanford)

Hakka Labs

DataEngConf SF16 - High cardinality time series search

Hakka Labs

DataEngConf SF16 - Data Asserts: Defensive Data Science

Hakka Labs

DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data

Hakka Labs

DataEngConf SF16 - Recommendations at Instacart

Hakka Labs

DataEngConf SF16 - Running simulations at scale

Hakka Labs

DataEngConf SF16 - Deriving Meaning from Wearable Sensor Data

Hakka Labs

DataEngConf SF16 - Collecting and Moving Data at Scale

Hakka Labs

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQ

Hakka Labs

DataEngConf SF16 - Unifying Real Time and Historical Analytics with the Lambd...

Hakka Labs

DataEngConf SF16 - Three lessons learned from building a production machine l...

Hakka Labs

DataEngConf SF16 - Scalable and Reliable Logging at Pinterest

Hakka Labs

DataEngConf SF16 - Bridging the gap between data science and data engineering

Hakka Labs

DataEngConf SF16 - Multi-temporal Data Structures

Hakka Labs

DataEngConf SF16 - Entity Resolution in Data Pipelines Using Spark

Hakka Labs

DataEngConf SF16 - Beginning with Ourselves

Hakka Labs

DataEngConf SF16 - Routing Billions of Analytics Events with High Deliverability

Hakka Labs

DataEngConf SF16 - Tales from the other side - What a hiring manager wish you...

Hakka Labs

DataEngConf SF16 - Methods for Content Relevance at LinkedIn

Hakka Labs

DataEngConf SF16 - Spark SQL Workshop

Hakka Labs

Mehr von Hakka Labs (20)

Always Valid Inference (Ramesh Johari, Stanford)

DataEngConf SF16 - High cardinality time series search

DataEngConf SF16 - Data Asserts: Defensive Data Science

DatEngConf SF16 - Apache Kudu: Fast Analytics on Fast Data

DataEngConf SF16 - Recommendations at Instacart

DataEngConf SF16 - Running simulations at scale

DataEngConf SF16 - Deriving Meaning from Wearable Sensor Data

DataEngConf SF16 - Collecting and Moving Data at Scale

DataEngConf SF16 - BYOMQ: Why We [re]Built IronMQ

DataEngConf SF16 - Unifying Real Time and Historical Analytics with the Lambd...

DataEngConf SF16 - Three lessons learned from building a production machine l...

DataEngConf SF16 - Scalable and Reliable Logging at Pinterest

DataEngConf SF16 - Bridging the gap between data science and data engineering

DataEngConf SF16 - Multi-temporal Data Structures

DataEngConf SF16 - Entity Resolution in Data Pipelines Using Spark

DataEngConf SF16 - Beginning with Ourselves

DataEngConf SF16 - Routing Billions of Analytics Events with High Deliverability

DataEngConf SF16 - Tales from the other side - What a hiring manager wish you...

DataEngConf SF16 - Methods for Content Relevance at LinkedIn

DataEngConf SF16 - Spark SQL Workshop

Kürzlich hochgeladen

Data spaces in distributed environments should be allowed to evolve in agile ways providing data space owners with large flexibility about which data they store. Agility and heterogeneity, however, jeopardize data exchanges because representations may build on varying ontologies and data consumers may not rely on the semantic correctness of their queries in the context of semantically heterogeneous, evolving data spaces. Graph data spaces are one example of a powerful model for representing and querying data whose semantics may change over time. To assert and enforce conditions on individual graph data spaces, shape languages (e.g SHACL) have been developed. We investigate the question of how querying and programming can be guarded by reasoning over SHACL constraints in a distributed setting and we sketch a picture of how a future landscape based on semantically heterogeneous data spaces might look like.

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Steffen Staab

Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf

kalichargn70th171

Foundation models are machine learning models which are easily capable of performing variable tasks on large and huge datasets. FMs have managed to get a lot of attention due to this feature of handling large datasets. It can do text generation, video editing to protein folding and robotics. In case we believe that FMs can help the hospitals and patients in any way, we need to perform some important evaluations, tests to test these assumptions. In this review, we take a walk through Fms and their evaluation regimes assumed clinical value. To clarify on this topic, we reviewed no less than 80 clinical FMs built from the EMR data. We added all the models trained on structured and unstructured data. We are referring to this combination of structured and unstructured EMR data or clinical data.

Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...

harshavardhanraghave

Investing in AI transformation today The modern business advantage: Uncovering deep insights with AI Organizations around the world have come to recognize AI as the transformative technology that enables them to gain real business advantage. AI’s ability to organize vast quantities of data allows those who implement it to uncover deep business insights, augment human expertise, drive operational efficiency, transform their products, and better serve their customers

Microsoft AI Transformation Partner Playbook.pdf

Willy Marroquin (WillyDevNET)

Test automation is a cornerstone of software development and quality assurance in today's rapidly evolving digital landscape. Its significance cannot be overstated. Businesses can enhance efficiency, productivity, and accelerate software delivery to market through automation, streamlining testing processes effectively. This comprehensive guide addresses the best practices for test automation in 2024. It offers a detailed checklist to empower you to optimize your automation efforts and maintain a competitive edge.

The Ultimate Test Automation Guide_ Best Practices and Tips.pdf

kalichargn70th171

How To Use Server-Side Rendering with Nuxt.js

Andolasoft Inc

A great deal of attention in medical devices has shifted towards cybersecurity with the ratification of section 524B of the FD&C act. This new law enables the FDA to enforce cybersecurity controls in any medical device that is capable of networked communications or that has software. In this webinar we will recap the process for managing vulnerabilities, identify categories of vulnerabilities and solutions and more.

The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...

ICS

Tech Tuesday-Harness the Power of Effective Resource Planning with OnePlan’s ...

OnePlan Solutions

In the past six months, the AI landscape has undergone a massive transformation, ushering in a new era of productivity with the latest in Large Language Models (LLMs) and AI technology. This deep dive unlocks how to: Create CustomGPT Models: No coding needed to tailor AI for your unique projects. Integrate your own data, including PDFs and Excel sheets, making information handling a breeze. Plus, discover how to call your own actions/integrations for even more personalized utility. Navigate Advanced Prompting: Overcome AI's memory limits and utilize Retrieval-Augmented Generation for accessing your personalized data, streamlining how you interact with AI. Stay Ahead with AI Trends: Peek into the evolving world of LLMs, featuring newcomers like Google Gemini, Anthropic Claude, Open Sora, and Twitter Grok, and understand what their advancements mean for your productivity. Witness Real-Life Transformations: Through examples and prompt demonstrations, see firsthand how these AI strategies revolutionize routine tasks, from data analysis to content creation. Learn to leverage image output and input for advanced practical use cases, adding a new dimension to your productivity toolkit. No previous coding or AI experience is needed for this talk. Stay ahead in the fast-evolving world of work. Embrace the AI revolution and transform your workflow with advanced LLM techniques. Join us to ensure you're not left behind in the productivity race.

AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques

VictorSzoltysek

A Secure and Reliable Document Management System is Essential.docx

ComplianceQuest1

Many specialized tools cater to distinct stages within the software development lifecycle (SDLC). These tools target various aspects of development, delivery, and operations, each with its unique strengths. Uniting these diverse testing needs into a single continuous testing platform presents several challenges. Such a platform must seamlessly integrate with various development tools and environments, accommodate different testing methodologies, and remain flexible to adapt to organizational processes and quality standards.

The Guide to Integrating Generative AI into Unified Continuous Testing Platfo...

kalichargn70th171

Unlocking the Future of AI Agents with Large Language Models

aagamshah0812

Diamond Application Development Crafting Solutions with Precision

SolGuruz

Looking to embark on a digital project in New York City? Choosing the ideal Laravel development partner is pivotal. Begin by defining your project requirements clearly. Assess potential partners' experience, expertise, and technical proficiency, checking portfolios and client testimonials. Effective communication and collaboration are paramount, so evaluate partners' communication styles and project management approaches. Consider long-term scalability and support options, and discuss pricing and contracts transparently. Lastly, trust your instincts when selecting a partner aligned with your vision and values.

How to Choose the Right Laravel Development Partner in New York City_compress...

software pro Development

HR Software Buyers Guide in 2024 - HRSoftware.com

Fatema Valibhai

Looking for an efficient way to manage your finances? Look no further than our money management app. With easy-to-use features, you can track your expenses, create budgets, and monitor your savings goals all in one place. Our app provides real-time updates on your spending habits and helps you make smarter financial decisions. Take control of your finances today with our user-friendly money management app.

Right Money Management App For Your Financial Goals

Jhone kinadey

introduction-to-automotive Andoid os-csimmonds-ndctechtown-2021.pdf

VishalKumarJha10

Azure Native Qumulo scales elastically for common High Performance Compute (HPC) workloads based on application requirements for: Financial Services, Automotive, Genomics / Life Sciences, Media and Entertainment, Energy, Oil and Gas, etc. Performance can be dialed UP (and back down) much higher than the examples shown here. These slides offer a glimpse into ANQ's HPC capabilities, although at a smaller scale. We invite YOU to do your own testing (with a free ANQ trial) and work with us to test your HPC workloads in Azure.

Azure_Native_Qumulo_High_Performance_Compute_Benchmarks.pdf

ryanfarris8

In an era where security concerns are paramount, the integration of artificial intelligence (AI) into CCTV cameras has revolutionized surveillance capabilities. One of the most significant advancements is the ability to achieve real-time threat detection, enabling immediate responses to potential security breaches. This blog explores how AI is reshaping surveillance through real-time threat detection and the implications of this technology.

Optimizing AI for immediate response in Smart CCTV

shikhaohhpro

Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live Booking Contact Details :- WhatsApp Chat :- [+91-9999965857 ] The Best Call Girls Delhi At Your Service Russian Call Girls Delhi Doing anything intimate with can be a wonderful way to unwind from life's stresses, while having some fun. These girls specialize in providing sexual pleasure that will satisfy your fetishes; from tease and seduce their clients to keeping it all confidential - these services are also available both install and outcall, making them great additions for parties or business events alike. Their expert sex skills include deep penetration, oral sex, cum eating and cum eating - always respecting your wishes as part of the experience (29-April-2024(PSS)

Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live

Call Girls In Delhi Whatsup 9873940964 Enjoy Unlimited Pleasure

Kürzlich hochgeladen (20)