Josh Wills, Head of Data Engineering, Slack at MLconf SF 2017

•

2 likes•889 views

This document discusses the author's experience over the last decade working in data science and data engineering roles. It describes how technology problems were initially focused on identifying patterns in data to solve problems like fraud detection. However, over time policy concerns have become more prominent as models have grown more complex. The author notes that as technology problems shrink, product and policy challenges around topics like algorithmic transparency and collective action have become increasingly important to address at scale.

Josh Wills
June 17, 2017
1
I Build The Black Box:
Grappling With Product and Policy

About Me
● Cloudera’s (Former)
Director of Data Science
● Slack’s (Former)
Director of Data
Engineering
● SLI Engineer @ Slack

Agenda
Technology
Product Policy

My Last Decade
Technology

Historical Perspective
● The Product Problem Was a Given
○ Identify fraud/spam; predict clicks on ads
● Policy Concerns Were Limited
○ Either already well-defined or essentially nonexistent
● Everything Was About the Data and the Tech
○ Basic algorithmic choices followed by an endless cycle of
feature engineering and experimentation

My First Day Back

Embeddings

A Seismic Shift

Keeping Up With The arXiv

Deep Architectures as Custom Hardware

AutoML as Cross-Platform Compiler

The Incredible Shrinking Technology Problems
Technology
Product Policy

The Data Product Problem

One Approach

Understanding Black Boxes

Black Boxes and Black Hats

Caring The Least At Scale

The Logic of Collective Action

Starting Small

Scaling Up

Scaling Way Up

Thank You!
22
slack.com/jobs

More Related Content

Similar to Josh Wills, Head of Data Engineering, Slack at MLconf SF 2017

NYC Open Data Meetup-- Thoughtworks chief data scientist talk

NYC Open Data Meetup-- Thoughtworks chief data scientist talk

NYC Open Data Meetup-- Thoughtworks chief data scientist talk

Vivian S. Zhang

AI Orange Belt - Session 4

AI Orange Belt - Session 4

AI Orange Belt - Session 4

Why Cloud Computing Projects Fail

Why Cloud Computing Projects Fail

Why Cloud Computing Projects Fail

David Linthicum

Career in Data Science (July 2017, DTLA)

Career in Data Science (July 2017, DTLA)

Career in Data Science (July 2017, DTLA)

This talk by Stefan Streichsbier, Co-Founder of GuardRails.io, provides a brief history of how development, operations and security testing have become highly complex. It continues to outline the key problems with traditional security solutions and why in 2020 companies around the world are still figuring out a good way to manage security as part of rapid development cycles. Specifically, the big challenge of introducing and fixing new security issues versus tackling the existing security dept of existing applications. To quote Bishop Desmond Tutu, “There comes a point where we need to stop just pulling people out of the river. We need to go upstream and find out why they’re falling in.” After setting the stage, the remainder of the talk will focus on the paradigm shift that security solutions have to incorporate in order to solve the problem of sustainably secure applications on all layers. This will explore how the elements of Speed, Just in time training, and Data science have to be leveraged to empower development teams around the globe to get ahead for once and finally become able to move fast and be safe at the same time. The 3 core takeaways for the audience are: 1.) Where security practices have gone wrong so far. 2.) What new technologies will cause a paradigm shift in how security is applied at scale. 3.) How security will look like in 5-10 years.

The Future of DevSecOps

The Future of DevSecOps

The Future of DevSecOps

Stefan Streichsbier

Lift+fing 09 Michael Shiloh slides with notes

Lift+fing 09 Michael Shiloh slides with notes

Lift+fing 09 Michael Shiloh slides with notes

Data-X-Sparse-v2

Data-X-Sparse-v2

Data-X-Sparse-v2

Agile Architecture (MAE slides with speaker notes)

Agile Architecture (MAE slides with speaker notes)

Agile Architecture (MAE slides with speaker notes)

Speaker: Venkatesh Umaashankar LinkedIn: https://www.linkedin.com/in/venkateshumaashankar/ What will be discussed? What is Data Science? Types of data scientists What makes a Data Science Team? Who are its members? Why does a DS team need Full Stack Developer? Who should lead the DS Team Building a Data Science team in a Startup Vs Enterprise Case studies on: Evolution Of Airbnb’s DS Team How Facebook on-boards DS team and trains them Apple’s Acqui-hiring Strategy to build DS team Spotify -‘Center of Excellence’ Model Who should attend? Managers Technical Leaders who want to get started with Data Science

Building successful data science teams

Building successful data science teams

Building successful data science teams

Venkatesh Umaashankar

Getting Started in Data Science

Getting Started in Data Science

Getting Started in Data Science

"Big Data" is term heard more and more in industry – but what does it really mean? There is a vagueness to the term reminiscent of that experienced in the early days of cloud computing. This has led to a number of implications for various industries and enterprises. These range from identifying the actual skills needed to recruit talent to articulating the requirements of a "big data" project. Secondary implications include difficulties in finding solutions that are appropriate to the problems at hand – versus solutions looking for problems. This presentation will take a look at Big Data and offer the audience with some considerations they may use immediately to assess the use of analytics in solving their problems. The talk begins with an idea of how big "Big Data" can be. This leads to an appreciation of how important "Management Questions" are to assessing analytic needs. The fields of data and analysis have become extremely important and impact nearly all facets of life and business. During the talk we will look at the two pillars of Big Data – Data Warehousing and Predictive Analytics. Then we will explore the open source tools and datasets available to NATO action officers to work in this domain. Use cases relevant to NATO will be explored with the purpose of show where analytics lies hidden within many of the day-to-day problems of enterprises. The presentation will close with a look at the future. Advances in the area of semantic technologies continue. The much acclaimed consultants at Gartner listed Big Data and Semantic Technologies as the first- and third-ranked top technology trends to modernize information management in the coming decade. They note there is an incredible value "locked inside all this ungoverned and underused information." HQ SACT can leverage this powerful analytic approach to capture requirement trends when establishing acquisition strategies, monitor Priority Shortfall Areas, prepare solicitations, and retrieve meaningful data from archives.

Big Data in NATO and Your Role

Big Data in NATO and Your Role

Big Data in NATO and Your Role

Bit by Bit: A Framework for Building Technological Competence as a Lawyer

Bit by Bit: A Framework for Building Technological Competence as a Lawyer

Bit by Bit: A Framework for Building Technological Competence as a Lawyer

DataScience_introduction.pdf

DataScience_introduction.pdf

DataScience_introduction.pdf

SouravBiswas747273

Zia Khan, Computer Systems Analyst and Data Scientist at LearningFuze Data Science tutorial is designed for people who are new to Data Science. This is a beginner level session so no prior coding or technical knowledge is required. Just bring your laptop with WiFi capability. The session starts with a review of what is data science, the amount of data we generate and how companies are using that data to get insight. We will pick a business use case, define the data science process, followed by hands-on lab using python and Jupyter notebook. During the hands-on portion we will work with pandas, numpy, matplotlib and sklearn modules and use a machine learning algorithm to approach the business use case.

Data Con LA 2022 - Intro to Data Science

Data Con LA 2022 - Intro to Data Science

Data Con LA 2022 - Intro to Data Science

A look back at how the practice of data science has evolved over the years, modern trends, and where it might be headed in the future. Starting from before anyone had the title "data scientist" on their resume, to the dawn of the cloud and big data, and the new tools and companies trying to push the state of the art forward. Finally, some wild speculation on where data science might be headed. Presentation given to Seattle Data Science Meetup on Friday July 24th 2015.

The Evolution of Data Science

The Evolution of Data Science

The Evolution of Data Science

Lift+FING Marseille, France

Lift+FING Marseille, France

Lift+FING Marseille, France

GTU GeekDay 2019 Limitations of Artificial Intelligence

GTU GeekDay 2019 Limitations of Artificial Intelligence

GTU GeekDay 2019 Limitations of Artificial Intelligence

AI and ML for Everyone

AI and ML for Everyone

AI and ML for Everyone

This examines the potential for the application of Design Science principles to the solution design process within solution architecture to improve the rigour and accuracy of solution designs. Design Science is the structured and systematic process for creating designs that resolve problems. It is concerned with the structured process for the acquisition and application of knowledge in relation to the problems to the resolved and the solution knowledge to be applied. The application of Design Science must be a means to an end – better solution quality – and not an end in itself – an incentive for the design function is to become large. Solution architecture requires a (changing) combination of technical, leadership, interpersonal skills, experience, analysis, appropriate creativity, reflection and intuition applied in a structured manner. Knowledge management – problem knowledge and solution knowledge – is at the core of the application of design science principles. Knowledge management requires good management of the solution architecture function.

Design Science and Solution Architecture

Design Science and Solution Architecture

Design Science and Solution Architecture

ABSTRACT: The ongoing big data revolution has revolutionized the way in which technology is used to empower new business segments like social networking and transform old business segments like traditional retail. However, the DNA that is used to build data processing platform is evolving quite rapidly. There is a plethora of competing tools, technologies, and “religion” for how to build state-of-the-art data analysis frameworks. In this talk, I will go over five ways to build scalable high-performance long-lasting data analysis frameworks in the wrong way. Surprisingly, the industry is full of examples of organization building frameworks in this “wrong” way. Since the “right” way to build a technology framework is dependent on the key business drivers, it is my hope that this talk will spur a discussion on what is the “right” way for Pinterest. The talk will focus on technologies including “data plumbing” (e.g. tools in the Hadoop ecosystem), and statistical modeling methods (e.g. R and Python). In this talk, I’ll try to connect to platform builders, data scientists, and business decision makers. BIO: Jignesh Patel is a Professor in Computer Sciences at the University of Wisconsin-Madison, where he also earned his Ph.D. He has worked in the area of databases (now fashionably called “big data”) for over two decades. He has won several best paper awards, and industry research awards. He is the recipient of the Wisconsin COW teaching award, and the U. Michigan College of Engineering Education Excellence Award. He has a strong interest in seeing research ideas transition to actual products. His Ph.D. thesis work was acquired by NCR/Teradata in 1997, and he also co-founded Locomatix -- a startup that built a platform to power real-time data-driven mobile services. Locomatix became part of Twitter in 2013. He is an ACM Distinguished Scientist and an IEEE Senior Member. He also serves on the board of Lands’ End, and advises a number of startups.

Five Ways To Do Data Analytics "The Wrong Way"

Five Ways To Do Data Analytics "The Wrong Way"

Five Ways To Do Data Analytics "The Wrong Way"

Discover Pinterest

Similar to Josh Wills, Head of Data Engineering, Slack at MLconf SF 2017 (20)

NYC Open Data Meetup-- Thoughtworks chief data scientist talk

NYC Open Data Meetup-- Thoughtworks chief data scientist talk

NYC Open Data Meetup-- Thoughtworks chief data scientist talk

AI Orange Belt - Session 4

AI Orange Belt - Session 4

AI Orange Belt - Session 4

Why Cloud Computing Projects Fail

Why Cloud Computing Projects Fail

Why Cloud Computing Projects Fail

Career in Data Science (July 2017, DTLA)

Career in Data Science (July 2017, DTLA)

Career in Data Science (July 2017, DTLA)

The Future of DevSecOps

The Future of DevSecOps

The Future of DevSecOps

Lift+fing 09 Michael Shiloh slides with notes

Lift+fing 09 Michael Shiloh slides with notes

Lift+fing 09 Michael Shiloh slides with notes

Data-X-Sparse-v2

Data-X-Sparse-v2

Data-X-Sparse-v2

Agile Architecture (MAE slides with speaker notes)

Agile Architecture (MAE slides with speaker notes)

Agile Architecture (MAE slides with speaker notes)

Building successful data science teams

Building successful data science teams

Building successful data science teams

Getting Started in Data Science

Getting Started in Data Science

Getting Started in Data Science

Big Data in NATO and Your Role

Big Data in NATO and Your Role

Big Data in NATO and Your Role

Bit by Bit: A Framework for Building Technological Competence as a Lawyer

Bit by Bit: A Framework for Building Technological Competence as a Lawyer

Bit by Bit: A Framework for Building Technological Competence as a Lawyer

DataScience_introduction.pdf

DataScience_introduction.pdf

DataScience_introduction.pdf

Data Con LA 2022 - Intro to Data Science

Data Con LA 2022 - Intro to Data Science

Data Con LA 2022 - Intro to Data Science

The Evolution of Data Science

The Evolution of Data Science

The Evolution of Data Science

Lift+FING Marseille, France

Lift+FING Marseille, France

Lift+FING Marseille, France

GTU GeekDay 2019 Limitations of Artificial Intelligence

GTU GeekDay 2019 Limitations of Artificial Intelligence

GTU GeekDay 2019 Limitations of Artificial Intelligence

AI and ML for Everyone

AI and ML for Everyone

AI and ML for Everyone

Design Science and Solution Architecture

Design Science and Solution Architecture

Design Science and Solution Architecture

Five Ways To Do Data Analytics "The Wrong Way"

Five Ways To Do Data Analytics "The Wrong Way"

Five Ways To Do Data Analytics "The Wrong Way"

More from MLconf

Understanding Human Impact: Social and Equity Assessments for AI Technologies Social and Equity Impact Assessments have broad applications but can be a useful tool to explore and mitigate for Machine Learning fairness issues and can be applied to product specific questions as a way to generate insights and learnings about users, as well as impacts on society broadly as a result of the deployment of new and emerging technologies. In this presentation, my goal is to advocate for and highlight the need to consult community and external stakeholder engagement to develop a new knowledge base and understanding of the human and social consequences of algorithmic decision making and to introduce principles, methods and process for these types of impact assessments.

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

The Brain’s Guide to Dealing with Context in Language Understanding Like the visual cortex, the regions of the brain involved in understanding language represent information hierarchically. But whereas the visual cortex organizes things into a spatial hierarchy, the language regions encode information into a hierarchy of timescale. This organization is key to our uniquely human ability to integrate semantic information across narratives. More and more, deep learning-based approaches to natural language understanding embrace models that incorporate contextual information at varying timescales. This has not only led to state-of-the art performance on many difficult natural language tasks, but also to breakthroughs in our understanding of brain activity. In this talk, we will discuss the important connection between language understanding and context at different timescales. We will explore how different deep learning architectures capture timescales in language and how closely their encodings mimic the brain. Along the way, we will uncover some surprising discoveries about what depth does and doesn’t buy you in deep recurrent neural networks. And we’ll describe a new, more flexible way to think about these architectures and ease design space exploration. Finally, we’ll discuss some of the exciting applications made possible by these breakthroughs.

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Applying Computer Vision to Reduce Contamination in the Recycling Stream With China’s recent refusal of most foreign recyclables, North American waste haulers are scrambling to figure out how to make on-shore recycling cost-effective in order to continue providing recycling services. Recyclables that were once being shipped to China for manual sorting are now primarily being redirected to landfills or incinerators. Without a solution, a nearly $5 billion annual recycling market could come to a halt. Purity in the recycling stream is key to this effort as contaminants in the stream can increase the cost of operations, damage equipment and reduce the ability to create pure commodities suitable for creating recycled goods. This market disruption as a result of China’s new regulations, however, provides us the chance to re-examine and improve our current disposal & collection habits with modern monitoring & artificial intelligence technology. Using images from our in-dumpster cameras, Compology has developed an ML-based process that helps identify, measure and alert for contaminants in recycling containers before they are picked-up, helping keep the recycling stream clean. Our convolutional neural network flags potential instances of contamination inside a dumpster, enabling garbage haulers to know which containers have the wrong type of material inside. This allows them to provide targeted, timely education, and when appropriate, assess fines, to improve recycling compliance at the businesses and residences they serve, helping keep recycling services financially viable. In this presentation, we will walk through our ML-based contamination measurement and scoring process by showing how Waste Management, a national waste hauler, has experienced 57% contamination reduction in nearly 2,000 containers over six months, This progress shows significant strides towards financially viable recycling services.

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Quantum Computing: a Treasure Hunt, not a Gold Rush Quantum computers promise a significant step up in computational power over conventional computers, but also suffer a number of counterintuitive limitations --- both in their computational model and in leading lab implementations. In this talk, we review how quantum computers compete with conventional computers and how conventional computers try to hold their ground. Then we outline what stands in the way of successful quantum ML applications.

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Data Labeling as Religious Experience One of the most common places to deploy a production machine learning systems is as a replacement for a legacy rules-based system that is having a hard time keeping up with new edge cases and requirements. I'll be walking through the process and tooling we used to help us design, train, and deploy a model to replace a set of static rules we had for handling invite spam at Slack, talk about what we learned, and discuss some problems to solve in order to make these migrations easier for everyone.

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Project GaitNet: Ushering in the ImageNet moment for human Gait kinematics The emergence of the upright human bipedal gait can be traced back 4 to 2.8 million years ago, to the now extinct hominin Australopithecus afarensis. Fine grained analysis of gait using the modern MEMS sensors found on all smartphones not just reveals a lot about the person’s orthopedic and neuromuscular health status, but also has enough idiosyncratic clues that it can be harnessed as a passive biometric. While there were many siloed attempts made by the machine learning community to model Bipedal Gait sensor data, these were done with small datasets oft collected in restricted academic environs. In this talk, we will introduce the ImageNet moment for human gait analysis by presenting 'Project GaitNet', the largest ever planet-sized motion sensor based human bipedal gait dataset ever curated. We’ll also present the associated state-of-the-art results in classifying humans harnessing novel deep neural architectures and the related success stories we have enjoyed in transfer-learning into disparate domains of human kinematics analysis.

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Machine Learning Methods in Detecting Alzheimer’s Disease from Speech and Language Alzheimer's disease affects millions of people worldwide, and it is important to predict the disease as early and as accurate as possible. In this talk, I will discuss development of novel ML models that help classifying healthy people from those who develop Alzheimer's, using short samples of human speech. As an input to the model, features of different modalities are extracted from speech audio samples and transcriptions: (1) syntactic measures, such as e.g. production rules extracted from syntactic parse trees, (2) lexical measures, such as e.g. features of lexical richness and complexity and lexical norms, and (3) acoustic measures, such as e.g. standard Mel-frequency cepstral coefficients. I will present the ML model that detects cognitive impairment by reaching agreement among modalities. The resulting model is able to achieve state of the art performance in both supervised and semi-supervised manner, using manual transcripts of human speech. Additionally, I will discuss potential limitations of any fully-automated speech-based Alzheimer's disease detection model, focusing mostly on the analysis of the impact of a not-so-accurate automatic speech recognition (ASR) on the classification performance. To illustrate this, I will present the experiments with controlled amounts of artificially generated ASR errors and explain how the deletion errors affect Alzheimer's detection performance the most, due to their impact on the features of syntactic and lexical complexity.

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Optimized Image Classification on the Cheap In this talk, we anchor on building an image classifier trained on the Stanford Cars dataset to evaluate two approaches to transfer learning -fine tuning and feature extraction- and the impact of hyperparameter optimization on these techniques. Once we define the most performant transfer learning technique for Stanford Cars, we will double the size of the dataset through image augmentation to boost the classifier’s performance. We will use Bayesian optimization to learn the hyperparameters associated with image transformations using the downstream image classifier’s performance as the guide. In conjunction with model performance, we will also focus on the features of these augmented images and the downstream implications for our image classifier. To both maximize model performance on a budget and explore the impact of optimization on these methods, we apply a particularly efficient implementation of Bayesian optimization to each of these architectures in this comparison. Our goal is to draw on a rigorous set of experimental results that can help us answer the question: how can resource-constrained teams make trade-offs between efficiency and effectiveness using pre-trained models?

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

The Importance of Modeling Data Collection Data sets used in machine learning are often collected in a systematically biased way - certain data points are more likely to be collected than others. We call this "observation bias". For example, in health care, we are more likely to see lab tests when the patient is feeling unwell than otherwise. Failing to account for observation bias can, of course, result in poor predictions on new data. By contrast, properly accounting for this bias allows us to make better use of the data we do have. In this presentation, we discuss practical and theoretical approaches to dealing with observation bias. When the nature of the bias is known, there are simple adjustments we can make to nonparametric function estimation techniques, such as Gaussian Process models. We also discuss the scenario where the data collection model is unknown. In this case, there are steps we can take to estimate it from observed data. Finally, we demonstrate that having a small subset of data points that are known to be collected at random - that is, in an unbiased way - can vastly improve our ability to account for observation bias in the rest of the data set. My hope is that attendees of this presentation will be aware of the perils of observation bias in their own work, and be equipped with tools to address it.

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

The Uncanny Valley of ML Every so often, the conundrum of the Uncanny Valley re-emerges as advanced technologies evolve from clearly experimental products to refined accepted technologies. We have seen its effects in robotics, computer graphics, and page load times. The debate of how to handle the new technology detracts from its benefits. When machine learning is added to human decision systems a similar effect can be measured in increased response time and decreased accuracy. These systems include radiology, judicial assignments, bus schedules, housing prices, power grids and a growing variety of applications. Unfortunately, the Uncanny Valley of ML can be hard to detect in these systems and can lead to degraded system performance when ML is introduced, at great expense. Here, we'll introduce key design principles for introducing ML into human decision systems to navigate around the Uncanny Valley and avoid its pitfalls.

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

Deep Learning Architectures for Semantic Relation Detection Tasks Recognizing and distinguishing specific semantic relations from other types of semantic relations is an essential part of language understanding systems. Identifying expressions with similar and contrasting meanings is valuable for NLP systems which go beyond recognizing semantic relatedness and require to identify specific semantic relations. In this talk, I will first present novel techniques for creating labelled datasets required for training deep learning models for classifying semantic relations between phrases. I will further present various neural network architectures that integrate morphological features into integrated path-based and distributional relation detection algorithms and demonstrate that this model outperforms state-of-the-art models in distinguishing semantic relations and is capable of efficiently handling multi-word expressions.

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Building an Incrementally Trained, Local Taste Aware, Global Deep Learned Recommender System Model At Netflix, our main goal is to maximize our members’ enjoyment of the selected show by minimizing the amount of time it takes for them to find it. We try to achieve this goal by personalizing almost all the aspects of our product -- from what shows to recommend, to how to present these shows and construct their home-pages to what images to select per show, among many other things. Everything is recommendations for us and as an applied Machine Learning group, we spend our time building models for personalization that will eventually increase the joy and satisfaction of our members. In this talk we will primarily focus our attention on a) making a global deep learned recommender model that is regional tastes and popularity aware and b) adapting this model to changing taste preferences as well as dynamic catalog availability. We will first go through some standard recommender system models that use Matrix Factorization and Topic Models and then compare and contrast them with more powerful and higher capacity deep learning based models such as sequence models that use recurrent neural networks. We will show what it entails to build a global model that is aware of regional taste preferences and catalog availability. We will show how models that are built on simple Maximum Likelihood principle fail to do that. We will then describe one solution that we have employed in order to enable the global deep learned models to focus their attention on capturing regional taste preferences and changing catalog.In the latter half of the talk, we will discuss how we do incremental learning of deep learned recommender system models. Why do we need to do that ? Everything changes with time. Users’ tastes change with time. What’s available on Netflix and what’s popular also change over time. Therefore, updating or improving recommendation systems over time is necessary to bring more joy to users. In addition to how we apply incremental learning, we will discuss some of the challenges we face involving large-scale data preparation, infrastructure setup for incremental model training as well as pipeline scheduling. The incremental training enables us to serve fresher models trained on fresher and larger amounts of data. This helps our recommender system to nicely and quickly adapt to catalog and users’ taste changes, and improve overall performance.

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Vito Ostuni - The Voice: New Challenges in a Zero UI World The adoption of voice-enabled devices has seen an explosive growth in the last few years and music consumption is among the most popular use cases. Music personalization and recommendation plays a major role at Pandora in providing a daily delightful listening experience for millions of users. In turn, providing the same perfectly tailored listening experience through these novel voice interfaces brings new interesting challenges and exciting opportunities. In this talk we will describe how we apply personalization and recommendation techniques in three common voice scenarios which can be defined in terms of request types: known-item, thematic, and broad open-ended. We will describe how we use deep learning slot filling techniques and query classification to interpret the user intent and identify the main concepts in the query. We will also present the differences and challenges regarding evaluation of voice powered recommendation systems. Since pure voice interfaces do not contain visual UI elements, relevance labels need to be inferred through implicit actions such as play time, query reformulations or other types of session level information. Another difference is that while the typical recommendation task corresponds to recommending a ranked list of items, a voice play request translates into a single item play action. Thus, some considerations about closed feedback loops need to be made. In summary, improving the quality of voice interactions in music services is a relatively new challenge and many exciting opportunities for breakthroughs still remain. There are many new aspects of recommendation system interfaces to address to bring a delightful and effortless experience for voice users. We will share a few open challenges to solve for the future.

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

More from MLconf (20)

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Ted Willke - The Brain’s Guide to Dealing with Context in Language Understanding

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold Rush

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Josh Wills - Data Labeling as Religious Experience

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

Meghana Ravikumar - Optimized Image Classification on the Cheap

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

Noam Finkelstein - The Importance of Modeling Data Collection

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

June Andrews - The Uncanny Valley of ML

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection Tasks

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Vito Ostuni - The Voice: New Challenges in a Zero UI World

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Anna choromanska - Data-driven Challenges in AI: Scale, Information Selection...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Janani Kalyanam - Machine Learning to Detect Illegal Online Sales of Prescrip...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Esperanza Lopez Aguilera - Using a Bayesian Neural Network in the Detection o...

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Neel Sundaresan - Teaching a machine to code

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Rishabh Mehrotra - Recommendations in a Marketplace: Personalizing Explainabl...

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Soumith Chintala - Increasing the Impact of AI Through Better Software

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

Roy Lowrance - Predicting Bond Prices: Regime Changes

Recently uploaded

In this session, we will showcase how to revolutionize automated testing for your software, automation, and QA teams with UiPath Test Suite. In part 1 of UiPath test automation using UiPath Test Suite – developer series, we will cover, Software testing overview What is software testing Why software testing is required Typical test types and levels Continuous testing and challenges Introduction to UiPath Test Suite UiPath Test Suite family of products Speaker: Atul Trikha, Chief Technologist & Solutions Architect, Peraton and UiPath MVP Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

UiPath Test Automation using UiPath Test Suite series, part 1

UiPath Test Automation using UiPath Test Suite series, part 1

UiPath Test Automation using UiPath Test Suite series, part 1

Intrigued by why some of the world's largest companies (Netflix, Google, Cisco, Twitter, Uber etc) are using gRPC? In this demo based talk we delve into the world of gRPC in .Net, what it does and why we should use it. We compare the interface with both Rest and graphQL. We will show you how to implement grpc server-side in .net and in the web. Finally, I will show you how the tooling helps you deliver powerful interfaces and interact with them quickly and simply.

Demystifying gRPC in .Net by John Staveley

Demystifying gRPC in .Net by John Staveley

Demystifying gRPC in .Net by John Staveley

Intro in Product Management - Коротко про професію продакт менеджера

Intro in Product Management - Коротко про професію продакт менеджера

Intro in Product Management - Коротко про професію продакт менеджера

AI presentation and introduction - Retrieval Augmented Generation RAG 101

AI presentation and introduction - Retrieval Augmented Generation RAG 101

AI presentation and introduction - Retrieval Augmented Generation RAG 101

The standard Salesforce Approval process can be limiting in many ways, especially in complex scenarios. What if there was a way to implement very flexible approvals where one can use Apex code to make data updates in unrelated records, dynamically generate next steps details, and compute assignees on the fly? And still use UI-based configurations to implement concrete approval processes. In this session, we will share ideas behind such a solution and show a few lines of code to get you started.

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Explore the core of Salesforce success in 'Salesforce Adoption – Metrics, Methods, and Motivation.' We will discuss essential metrics, effective methods to drive adoption, and the driving force behind user engagement and explore strategies for onboarding, training, and continuous support that empower users to navigate the platform seamlessly. By leveraging these tools, you can effectively measure adoption against your company’s goals and create an environment where users not only adopt Salesforce but actively contribute to its ongoing success.

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom

I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.

"Impact of front-end architecture on development cost", Viktor Turskyi

"Impact of front-end architecture on development cost", Viktor Turskyi

"Impact of front-end architecture on development cost", Viktor Turskyi

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

ScyllaDB has the potential to deliver impressive performance and scalability. The better you understand how it works, the more you can squeeze out of it. But before you squeeze, make sure you know what to monitor! Watch our experienced Postgres developer work through monitoring and performance strategies that help him understand what mistakes he’s made moving to NoSQL. And learn with him as our database performance expert offers friendly guidance on how to use monitoring and performance tuning to get his sample Rust application on the right track. This webinar focuses on using monitoring and performance tuning to discover and correct mistakes that commonly occur when developers move from SQL to NoSQL. For example: - Common issues getting up and running with the monitoring stack - Using the CQL optimizations dashboard - Common issues causing high latency in a node - Common issues causing replica imbalance - What a healthy system looks like in terms of memory - Key metrics to keep an eye on This isn’t “Death-by-Powerpoint.” We’ll walk through problems encountered while migrating a real application from Postgres to ScyllaDB – and try to fix them live as well.

Optimizing NoSQL Performance Through Observability

Optimizing NoSQL Performance Through Observability

Optimizing NoSQL Performance Through Observability

New customer? New industry? New cloud? New team? A lot to handle! How to ensure the success of the project? Start it well! I've created the 3 areas of focus at the beginning of the project that helped me in multiple roles (BA, PO, and Consultant). Learn from real-world experiences and discover how these insights can empower you to deliver unparalleled value to your customers right from the project's start.

Powerful Start- the Key to Project Success, Barbara Laskowska

Powerful Start- the Key to Project Success, Barbara Laskowska

Powerful Start- the Key to Project Success, Barbara Laskowska

This talk focuses on the practical aspects of integrating various telephony systems with Salesforce, drawing on examples from implementations in the Czech scene. It aims to inform attendees about the spectrum of telephony solutions available, from small to large scale, and their compatibility with Salesforce. The presentation will highlight key considerations for selecting a telephony provider that integrates smoothly with Salesforce, including important questions to support the decision-making process. It will also discuss methods for integrating existing telephony systems with Salesforce, aimed at companies contemplating or in the process of adopting this CRM platform. The discussion is designed to provide a straightforward overview of the steps and considerations involved in telephony and Salesforce integration, with an emphasis on functionality, compatibility, and the practical experiences of Czech companies.

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

IoT Analytics Company Presentation May 2024

IoT Analytics Company Presentation May 2024

IoT Analytics Company Presentation May 2024

IESVE for Early Stage Design and Planning

IESVE for Early Stage Design and Planning

IESVE for Early Stage Design and Planning

Ever caught yourself nodding along when someone mentions "delivering value" in Agile, but secretly wondering what the heck they actually mean? You're not alone! Join us for an eye-opening session where we'll strip away the buzzwords and dive into the heart of Agile—value delivery. But what is "value"? Is it a mythical unicorn in the world of software development, or is there more to this overused term? This isn't going to be a sit-and-get lecture. We're talking about a face-to-face, interactive meetup where YOU play a crucial role. Come along to: Define It: What does "value" really mean? We’ll build a definition that’s not just words, but a compass for your Agile journey. Contextualise It: Discover what value means specifically to you, your team, your company, and your industry. Because one size does not fit all. Deliver It: Share strategies and gather new ones for uncovering and delivering true value—no more shooting in the dark!

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

Already know how to write a basic SOQL query? Great! But what about an *aggregate* SOQL query? You know, the kind that uses aggregate functions like COUNT & MAX along with GROUP BY and HAVING clauses? No? Well, get ready to learn how to slice & dice your org’s data right inside your own dev console. From finding duplicate records to prototyping summary & matrix reports, learn the ins and outs of aggregate queries during this fast-paced but admin-friendly session on advanced SOQL concepts.

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...

Enterprise Security Monitoring, And Log Management.

Enterprise Security Monitoring, And Log Management.

Enterprise Security Monitoring, And Log Management.

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams. Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Jeffrey Haguewood

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

When you think of a highly secure meeting environment, do you instantly think 'Microsoft Teams'!? Or do you think about some unknown application, troublesome UI and daunting login process...? If you think the latter - let's change that! In this session Femke will show you how using Teams Premium features can create secure, but also good looking meetings! PRETTY. Make sure your company's brand is represented before, during and after the meeting with Customization policies in place. SECURE. Lets utilize Meeting templates and Sensitivity Labels to protect your meeting and data to prevent sensitive information from being leaked. After this session, you will have a clear understanding of the capabilities of Teams Premium features and how to set up the perfect meeting that suits your organizational requirements!

ECS 2024 Teams Premium - Pretty Secure

ECS 2024 Teams Premium - Pretty Secure

ECS 2024 Teams Premium - Pretty Secure

Femke de Vroome

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 1

UiPath Test Automation using UiPath Test Suite series, part 1

UiPath Test Automation using UiPath Test Suite series, part 1

Demystifying gRPC in .Net by John Staveley

Demystifying gRPC in .Net by John Staveley

Demystifying gRPC in .Net by John Staveley

Intro in Product Management - Коротко про професію продакт менеджера

Intro in Product Management - Коротко про професію продакт менеджера

Intro in Product Management - Коротко про професію продакт менеджера

AI presentation and introduction - Retrieval Augmented Generation RAG 101

AI presentation and introduction - Retrieval Augmented Generation RAG 101

AI presentation and introduction - Retrieval Augmented Generation RAG 101

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom

Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom

"Impact of front-end architecture on development cost", Viktor Turskyi

"Impact of front-end architecture on development cost", Viktor Turskyi

"Impact of front-end architecture on development cost", Viktor Turskyi

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Optimizing NoSQL Performance Through Observability

Optimizing NoSQL Performance Through Observability

Optimizing NoSQL Performance Through Observability

Powerful Start- the Key to Project Success, Barbara Laskowska

Powerful Start- the Key to Project Success, Barbara Laskowska

Powerful Start- the Key to Project Success, Barbara Laskowska

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

Integrating Telephony Systems with Salesforce: Insights and Considerations, B...

IoT Analytics Company Presentation May 2024

IoT Analytics Company Presentation May 2024

IoT Analytics Company Presentation May 2024

IESVE for Early Stage Design and Planning

IESVE for Early Stage Design and Planning

IESVE for Early Stage Design and Planning

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...

Enterprise Security Monitoring, And Log Management.

Enterprise Security Monitoring, And Log Management.

Enterprise Security Monitoring, And Log Management.

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

ECS 2024 Teams Premium - Pretty Secure

ECS 2024 Teams Premium - Pretty Secure

ECS 2024 Teams Premium - Pretty Secure

Josh Wills, Head of Data Engineering, Slack at MLconf SF 2017

1. Josh Wills June 17, 2017 1 I Build The Black Box: Grappling With Product and Policy

2. About Me ● Cloudera’s (Former) Director of Data Science ● Slack’s (Former) Director of Data Engineering ● SLI Engineer @ Slack

3. Agenda Technology Product Policy

4. My Last Decade Technology

5. Historical Perspective ● The Product Problem Was a Given ○ Identify fraud/spam; predict clicks on ads ● Policy Concerns Were Limited ○ Either already well-defined or essentially nonexistent ● Everything Was About the Data and the Tech ○ Basic algorithmic choices followed by an endless cycle of feature engineering and experimentation

6. My First Day Back

8. A Seismic Shift

9. Keeping Up With The arXiv

10. Deep Architectures as Custom Hardware

11. AutoML as Cross-Platform Compiler

12. The Incredible Shrinking Technology Problems Technology Product Policy

13. The Data Product Problem

14. One Approach

15. Understanding Black Boxes

16. Black Boxes and Black Hats

17. Caring The Least At Scale

18. The Logic of Collective Action

19. Starting Small

21. Scaling Way Up

22. Thank You! 22 slack.com/jobs