Better Search Through Query Understanding

•Als PPTX, PDF herunterladen•

44 gefällt mir•31,645 views

This document discusses query understanding in search engines. It describes how query understanding involves identifying entities and tags in queries, predicting the user's intent or topic area, expanding queries using related terms, and incorporating spelling corrections. The key aspects of query understanding covered are tagging queries for entities like names, titles, companies; predicting the user's vertical intent like jobs, people or companies; and expanding queries using name synonyms, job title synonyms or signals from past user queries and clicks. The document also suggests giving users more transparency, guidance and control over the search process.

Technologie Business

Recruiting SolutionsRecruiting SolutionsRecruiting Solutions
Daniel Tunkelang
Head, Query Understanding
better search through
query understanding

overview
 query understanding: what is it?
 how we do query understanding at LinkedIn
 some other thoughts from search in the wild
what I’m not going to cover:
2

Information need query select from results
rank using IR model
user:
system:
tf-idf PageRank
bird’s-eye view of how a search engine works
3

Information need query select from results
rank using IR model
user:
system:
tf-idf PageRank
query understanding
4

6
tag: skill OR title
related skills:
search, ranking, …
tag: company
id: 1337
industry: internet
verticals:
people, jobs
intent: exploratory

query understanding pipeline
7
spellcheck
query tagging
vertical intent prediction
query expansion
raw query
structured query
+
annotations

query understanding pipeline
8
spellcheck
query tagging
vertical intent prediction
query expansion
raw query
structured query
+
annotations

9
fix obvious typos
help users spell names
spelling correction

spelling out the details
10
PEOPLE NAMES
COMPANIES
TITLES
PAST QUERIES
n-grams
marissa => ma ar ri is ss sa
metaphone
mark/marc => MRK
co-occurrence counts
marissa:mayer = 1000
marisa meyer yahoo
marissa
marisa
meyer
mayer
yahoo

spelling out the details
11
problem: corpus as well as query logs contain many spelling errors
certain spelling errors are quite frequent
while genuine words (especially names) might be infrequent

spelling out the details
12
problem: corpus & query logs contain spelling errors
solution: use query chains to infer correct spelling
[product manger] [product manager] CLICK
[marissa mayer] CLICK

query understanding pipeline
13
spellcheck
query tagging
vertical intent prediction
query expansion
raw query
structured query
+
annotations

query tagging: identifying entities in the query
14
TITLE CO GEO
TITLE-237
software engineer
software developer
programmer
…
CO-1441
Google Inc.
Industry: Internet
GEO-7583
Country: US
Lat: 42.3482 N
Long: 75.1890 W
(RECOGNIZED TAGS: NAME, TITLE, COMPANY, SCHOOL, GEO, SKILL )

query tagging: identifying entities in the query
15
TITLE CO GEO
MORE PRECISE MATCHING WITH DOCUMENTS

query tagging: sequential model
22
EMISSION PROBABILITIES
(learned from user profiles)
TRANSITION PROBABILITIES
(learned from query logs)
TRAINING

query tagging: sequential model
23
INFERENCE
given a query, find the most likely sequence of tags

query understanding pipeline
24
spellcheck
query tagging
vertical intent prediction
query expansion
raw query
structured query
+
annotations

vertical intent prediction: distribution
25
JOBS
PEOPLE
COMPANIES
(probability distribution over verticals)

vertical intent prediction: relevance
26
[company]
[employees]
[jobs]
[name search]

query understanding pipeline
27
spellcheck
query tagging
vertical intent prediction
query expansion
raw query
structured query
+
annotations

30
query expansion: signals
[jon] [jonathan] CLICK
trained using query chains:
[programmer] [developer] CLICK
symmetric but not transitive!
[francis] ⇔ [frank]
[franklin] ⇔ [frank]
[francis] ≠ [franklin]
[software engineer] [software developer] CLICK
context based!
[software engineer] => [software developer]
[civil engineer] ≠ [civil developer]

query understanding pipeline
31
spellcheck
query tagging
vertical intent prediction
query expansion
raw query
structured query
+
annotations

32
what else can we learn from search in the wild?

don’t guess when it’s better to ask
33
vs.

give users transparency, guidance, and control
35

think beyond individual search queries
36
Gene Golovchinsky, FXPAL

know when you don’t know
37
Claudia Hauff, Query Difficulty for Digital Libraries [2009]

38
Daniel Tunkelang
dtunkelang@linkedin.com
https://linkedin.com/in/dtunkelang

Weitere ähnliche Inhalte

Was ist angesagt?

Making Netflix Machine Learning Algorithms Reliable

Justin Basilico

Deep Learning for Personalized Search and Recommender Systems

Benjamin Le

These are the slides of my invited talk at the REVEAL workshop at RecSys 2019. The workshop focuses on the offline evaluation for recommender systems, and this year’s focus was on Reinforcement Learning. Although not directly related to reinforcement learning, it is clear that there are connections to what research in reinforcement learning is attempting to achieve (defining the rewards) and metrics that are optimized by recommender systems. I presented various works and personal thoughts on how to develop metrics of user engagement, which recommender systems can optimize for. An important message was that, for recommender systems to work both in the short and the long-term, it is important to consider the heterogeneity of both user and content to formalise the notion of engagement, and in turn design the appropriate metrics to capture these and optimize for. One way to achieve this is to follow these four steps: 1) Understanding intents; 2) Optimizing for the right metric; 3) Acting on segmentation; and 4) Thinking about diversity. An previous version of this talk was given to UMAP 2019. See https://www.slideshare.net/mounialalmas/metrics-engagement-personalization

Engagement, metrics and "recommenders"

Mounia Lalmas-Roelleke

As a leading e-commerce company in fashion in the Netherlands, Wehkamp dedicates itself to provide a better shopping experience for the customers. Using Spark, the data science team is able to develop various machine-learning projects for this purpose based on the large scale data of products and customers. A major topic for the data science team is ranking products. If a visitor enters a search phrase, what are the best products that fit the search phrase and in what order should the products been shown? Ranking products is also important if a visitor enters a product overview page, where hundreds or even thousands of products of a certain article type are displayed. In this project, Spark is used in the whole pipeline: retrieving and processing the search phrases and their results, making click models, creating feature sets, training and evaluating ranking models, pushing the models to production using ElasticSearch and creating Tableau dashboarding. In this talk, we are going to demonstrate how we use Spark to build up the whole pipeline of ranking products and the challenges we faced along the way.

Applied Machine Learning for Ranking Products in an Ecommerce Setting

Databricks

Homepage Personalization at Spotify

Oguz Semerci

Find and Be Found: Information Retrieval at LinkedIn SIGIR 2013 Industry Track Presentation http://sigir2013.ie/industry_track.html LinkedIn has a unique data collection: the 200M+ members who use LinkedIn are also the most valuable entities in our corpus, which consists of people, companies, jobs, and a rich content ecosystem. Our members use LinkedIn to satisfy a diverse set of navigational and exploratory information needs, which we address by leveraging semi-structured and social content to understanding their query intent and deliver a personalized search experience. In this talk, we will discuss some of the unique challenges we face in building the LinkedIn search platform, the solutions we've developed so far, and the open problems we see ahead of us. Shakti Sinha heads LinkedIn's search relevance team, and has been making key contributions to LinkedIn's search products since 2010. He previously worked at Google as both a research intern and a software engineer. He has an MS in Computer Science from Stanford, as well as a BS degree from College of Engineering, Pune. Daniel Tunkelang leads LinkedIn's efforts around query understanding. Before that, he led LinkedIn's product data science team. He previously led a local search quality team at Google and was a founding employee of Endeca (acquired by Oracle in 2011). He has written a textbook on faceted search, and is a recognized advocate of human-computer interaction and information retrieval (HCIR). He has a PhD in Computer Science from CMU, as well as BS and MS degrees from MIT.

Find and be Found: Information Retrieval at LinkedIn

Daniel Tunkelang

Personalized Page Generation for Browsing Recommendations

Justin Basilico

Past, Present & Future of Recommender Systems: An Industry Perspective

Justin Basilico

Incorporating Diversity in a Learning to Rank Recommender System

Jacek Wasilewski

Search@airbnb

Mousom Gupta

What’s in a Query? Understanding query intent

FlipkartStories

Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial

Alexandros Karatzoglou

Recommendation system

Akshat Thakar

These are the slides of a talk about some of our research at Spotify, as part of the celebration kickoff of Chalmers AI Research Centre in Gothenburg. I always like to make a story in my talk, and this time I wanted to reflect on the "push" (think recommender system) and "pull" (think search) paradigms. I am using this quote from Nicholas Belkin and Bruce Croft from their Communications of the ACM article published in 1992 to frame my story: "We conclude that information retrieval and information filtering are indeed two sides of the same coin. They work together to help people get the information needed to perform their tasks."

Recommending and Searching (Research @ Spotify)

Mounia Lalmas-Roelleke

The “People You May Know” (PYMK) recommendation service helps LinkedIn’s members identify other members that they might want to connect to and is the major driver for growing LinkedIn's social network. The principal challenge in developing a service like PYMK is dealing with the sheer scale of computation needed to make precise recommendations with a high recall. PYMK service at LinkedIn has been operational for over a decade, during which it has evolved from an Oracle-backed system that took weeks to compute recommendations to a Hadoop backed system that took a few days to compute recommendations to its most modern embodiment where it can compute recommendations in near real time. This talk will present the evolution of PYMK to its current architecture. We will focus on various systems we built along the way, with an emphasis on systems we built for our most recent architecture, namely Gaia, our real-time graph computing capability, and Venice our online feature store with scoring capability, and how we integrate these individual systems to generate recommendations in a timely and agile manner, while still being cost-efficient. We will briefly talk about the lessons learned about scalability limits of our past and current design choices and how we plan to tackle the scalability challenges for the next phase of growth. https://qcon.ai/qconai2019/presentation/people-you-may-know-fast-recommendations-over-massive-data

[QCon.ai 2019] People You May Know: Fast Recommendations Over Massive Data

Sumit Rangwala

X-Ray Searching - SourceBreaker

Hunted

Tutorial on Question Answering Systems

Saeedeh Shekarpour

Boolean Training

Somil Charan

ML Infra for Netflix Recommendations - AI NEXTCon talk

Faisal Siddiqi

Learned word embeddings such as Word2vec/Glove were initially found to be effective for broad range of tasks in Natural Language Processing (NLP). More recently though, these are being used successfully in areas well beyond text such as graphs and event streams. In this talk Sharath will speak about how we use learned embeddings at Instacart for search ranking, personalization and product recommendations. Presented at: SF Data Mining Meetup https://www.meetup.com/Data-Mining/events/237164197/

Learned Embeddings for Search and Discovery at Instacart

Sharath Rao

Was ist angesagt? (20)

Making Netflix Machine Learning Algorithms Reliable

Deep Learning for Personalized Search and Recommender Systems

Engagement, metrics and "recommenders"

Applied Machine Learning for Ranking Products in an Ecommerce Setting

Homepage Personalization at Spotify

Find and be Found: Information Retrieval at LinkedIn

Personalized Page Generation for Browsing Recommendations

Past, Present & Future of Recommender Systems: An Industry Perspective

Incorporating Diversity in a Learning to Rank Recommender System

Search@airbnb

What’s in a Query? Understanding query intent

Learning to Rank for Recommender Systems - ACM RecSys 2013 tutorial

Recommendation system

Recommending and Searching (Research @ Spotify)

[QCon.ai 2019] People You May Know: Fast Recommendations Over Massive Data

X-Ray Searching - SourceBreaker

Tutorial on Question Answering Systems

Boolean Training

ML Infra for Netflix Recommendations - AI NEXTCon talk

Learned Embeddings for Search and Discovery at Instacart

Ähnlich wie Better Search Through Query Understanding

Hr analytics and graphs : job recommendations

Linkurious

Next generation linked in talent search

Ryan Wu

AI, Search, and the Disruption of Knowledge Management

Trey Grainger

Skills, Reputation, and Search

Peter Skomoroch

Keynote Peter Skomoroch - skills, reputation, and search

lucenerevolution

KEYNOTE: Skills, Reputation and Search

lucenerevolution

Learn to Rank search results

Ganesh Venkataraman

Neo4j: Data Engineering for RAG (retrieval augmented generation)

Neo4j

Path 101 Opportunity

path101

Building a semantic search system - one that can correctly parse and interpret end-user intent and return the ideal results for users’ queries - is not an easy task. It requires semantically parsing the terms, phrases, and structure within queries, disambiguating polysemous terms, correcting misspellings, expanding to conceptually synonymous or related concepts, and rewriting queries in a way that maps the correct interpretation of each end user’s query into the ideal representation of features and weights that will return the best results for that user. Not only that, but the above must often be done within the confines of a very specific domain - ripe with its own jargon and linguistic and conceptual nuances. This talk will walk through the anatomy of a semantic search system and how each of the pieces described above fit together to deliver a final solution. We'll leverage several recently-released capabilities in Apache Solr (the Semantic Knowledge Graph, Solr Text Tagger, Statistical Phrase Identifier) and Lucidworks Fusion (query log mining, misspelling job, word2vec job, query pipelines, relevancy experiment backtesting) to show you an end-to-end working Semantic Search system that can automatically learn the nuances of any domain and deliver a substantially more relevant search experience.

How to Build a Semantic Search System

Trey Grainger

In this presentation, delivered by ABK Andreas Kollegger at QCon London 2024, the focus was on Connecting the Dots for Information Discovery. The classic RAG application extends an LLM with private information, able to fetch answers to questions that are contained in a single chunk of text. What if the answer requires connecting the dots across multiple chunks that may not be directly similar to the question? That is information discovery with GraphRAG. You'll learn how to: - reconstruct chunks into the original context - meaningfully connect disparate chunks - expand unstructured text data with structured data - combine all this into a RAG workflow

Connecting the Dots for Information Discovery.pdf

Neo4j

Search Ranking Across Heterogeneous Information Sources

Viet Ha-Thuc

Hello Sir/Madam, Please find enclosed my CV in application for the Digital Marketing Executive The nature of my degree course has prepared me for this position. It involved a great deal of independent research, requiring initiative, self-motivation and a wide range of skills. I am, an understanding of the online industry was essential. I found this subject very stimulating. Thank you for taking the time to consider this application and I look forward to hearing from you in the near future. Yours sincerely Alok Das Current CTC -3.35 lacs/year Expected CTC - 5 lacs/year Immediate Joining in 10 days

Alok das cv updated

Alok Das

A Recruiter's Checklist (Strings Attached)

Maysoun Mohamed

Activate 2019 Opening Keynote, Will Hayes, CEO, Lucidworks

Lucidworks

In this presentation, we’ll talk about evolving self-learning search and recommendation systems which are able to accept user queries, deliver relevance-ranked results, and iteratively learn from the users’ subsequent interactions to continually deliver a more relevant experience. Such a self-learning system leverages reflected intelligence to consistently improve its understanding of the content (documents and queries), the context of specific users, and the collective feedback from all prior user interactions with the system. Through iterative feedback loops, such a system can leverage user interactions to learn the meaning of important phrases and topics within a domain, identify alternate spellings and disambiguate multiple meanings of those phrases, learn the conceptual relationships between phrases, and even learn the relative importance of features to automatically optimize its own ranking algorithms on a per-query, per-category, or per-user/group basis.

Reflected intelligence evolving self-learning data systems

Trey Grainger

Gary Conaway 6-21-11

Melanie Brandt

You Don't Know SEO

Michael King

Shall we search? Lviv.

Vira Povkh

Relational databases were conceived to digitize paper forms and automate well-structured business processes, and still have their uses. But, oftentimes with RDBMS, performance degrades with the increasing number and levels of data relationships and data size. A graph database like Neo4j naturally stores, manages, analyzes, and uses data within the context of connections meaning Neo4j provides faster query performance and vastly improved flexibility in handling complex hierarchies than SQL. This webinar explains why companies are shifting away from RDBMS towards graphs to unlock the business value in their data relationships.

RDBMS to Graph Webinar

Neo4j

Ähnlich wie Better Search Through Query Understanding (20)

Hr analytics and graphs : job recommendations

Next generation linked in talent search

AI, Search, and the Disruption of Knowledge Management

Skills, Reputation, and Search

Keynote Peter Skomoroch - skills, reputation, and search

KEYNOTE: Skills, Reputation and Search

Learn to Rank search results

Neo4j: Data Engineering for RAG (retrieval augmented generation)

Path 101 Opportunity

How to Build a Semantic Search System

Connecting the Dots for Information Discovery.pdf

Search Ranking Across Heterogeneous Information Sources

Alok das cv updated

A Recruiter's Checklist (Strings Attached)

Activate 2019 Opening Keynote, Will Hayes, CEO, Lucidworks

Reflected intelligence evolving self-learning data systems

Gary Conaway 6-21-11

You Don't Know SEO

Shall we search? Lviv.

RDBMS to Graph Webinar

Mehr von Daniel Tunkelang

Query Understanding and Ecommerce

Daniel Tunkelang

Semantic Equivalence of e-Commerce Queries

Daniel Tunkelang

Behavioral economics transformed how we think about human decision making, rejecting expected utility maximization for the real world of heuristics, biases, and satisficing. In this talk, I'll argue that our thinking about search engines needs a similar transformation. I will compare the Probability Ranking Principle to expected utility maximization and offer ways that AI can help searchers satisfice through query understanding. This was an invited talk given at the 2023 Walmart AI Summit. Speaker Bio Daniel Tunkelang is an independent consultant specializing in search, machine learning / AI, and data science. He completed undergraduate and master's degrees in Computer Science and Math at MIT and a PhD in computer science at CMU. He was a founding employee and chief scientist of Endeca, a search pioneer that Oracle acquired in 2011. He then led engineering and data science teams at Google and LinkedIn. He has written a book on Faceted Search, and he blogs on Medium about search-related topics — particularly query understanding. He has worked with numerous tech companies, retailers, and others, including Algolia, Apple, Canva, Coupang, eBay, Etsy, Flipkart, Home Depot, Oracle, Pinterest, Salesforce, Target, Yelp, and Zoom.

Helping Searchers Satisfice through Query Understanding

Daniel Tunkelang

MMM, Search! An opinionated discussion of search metrics, models, and methods. Presented to the Wikimedia Foundation on April 27, 2020. About the Speaker Daniel Tunkelang is an independent consultant specializing in search, discovery, machine learning / AI, and data science. He was a founding employee of Endeca, a search pioneer that Oracle acquired. After 10 years at Endeca, he moved to Google, where he led a local search team. He then served as a director of data science and search at LinkedIn. After leaving LinkedIn in 2015, he became an independent consultant. His clients have included Apple, eBay, Coupang, Etsy, Flipkart, Gartner, Pinterest, Salesforce, and Yelp; as well as some of the largest traditional retailers. Daniel completed undergraduate and master's degrees in Computer Science and Math at MIT and a Ph.D. in computer science at CMU. He wrote a book on Faceted Search, published by Morgan & Claypool, and he blogs on Medium about search-related topics -- particularly about query understanding. He is also active on Twitter, LinkedIn, and Quora.

MMM, Search!

Daniel Tunkelang

Enterprise Intelligence: Putting the Pieces Together http://enterpriserelevance.com/kdd2016/keynote.html These slides are for a keynote presentation delivered at the Workshop on Enterprise Intelligence, held in conjunction with the 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2016). About the author: Daniel Tunkelang is a data science and engineering executive who has built and led some of the strongest teams in the software industry. He studied computer science and math at MIT and has a PhD in computer science from CMU. He was a founding employee and chief scientist of Endeca, a search pioneer that Oracle acquired for $1.1B. He led a local search team at Google. He was a director of data science and engineering at LinkedIn, and he established their query understanding team. Daniel is a widely recognized writer and speaker. He is frequently invited to speak at academic and industry conferences, particularly in the areas of information retrieval, web science, and data science. He has written the definitive textbook on faceted search (now a standard for ecommerce sites), established an annual symposium on human-computer interaction and information retrieval, and authored 24 US patents. His social media posts have attracted over a million page views. Daniel advises and consults for companies that can benefit strategically from his expertise. His clients range from early-stage startups to "unicorn" technology companies like Etsy and Pinterest. He helps companies make decisions around algorithms, technology, product strategy, hiring, and organizational structure.

Enterprise Intelligence

Daniel Tunkelang

Query Understanding: A Manifesto

Daniel Tunkelang

Where should you put your data scientists?

Daniel Tunkelang

Data Science: A Mindset for Productivity Keynote at 2015 Ronin Labs West Coast CTO Summit https://www.eventjoy.com/e/west-coast-cto-summit-2015 Abstract Data science isn't just about using a collection of technologies and algorithms. Data science requires a mindset that solves problems at a higher level of abstraction. How do we model utility when we think about optimization? How do we decide which hypotheses to test? How do we allocate our scarce resources to make progress? There are no silver bullets. But I'll share what I've learned from a variety of contexts over the course of my work at Endeca, Google, and LinkedIn; and I hope you'll leave this talk with some practical wisdom you can apply to your next data science project.

Data Science: A Mindset for Productivity

Daniel Tunkelang

My Three Ex’s: A Data Science Approach for Applied Machine Learning Daniel Tunkelang (LinkedIn) Presented at QCon San Francisco 2014 in the Applied Machine Learning and Data Science track https://qconsf.com/presentation/my-three-ex%E2%80%99s-data-science-approach-applied-machine-learning Abstract This talk is about applying machine learning to solve problems. It’s not a talk about machine learning — or at least not about the theory of machine learning. Theoretical machine learning requires a deep understanding of computer science and statistics. It’s one of the most studied areas of computer science, and advances in theoretical machine learning give us hope of solving the world’s “AI-hard” problems. Applied machine learning is more grounded but no less important. We are surrounded by opportunities to apply classifiers, learn rules, compute similarity, and assemble clusters. We don’t need to develop new algorithms for any of these problems — our textbooks and open-source libraries have done that hard work for us. But algorithms are not enough. Applying machine learning to solve problems requires a data science mindset that transcends the algorithmic details. In this talk, I’ll communicate the data science mindset by describing my three ex’s: express, explain, and experiment. These three activities are the pillars of a successful strategy for applying machine learning to solve problems. Whether you’re a machine learning novice or expert, I hope you’ll leave this talk with some practical wisdom you can apply to your next project.

My Three Ex’s: A Data Science Approach for Applied Machine Learning

Daniel Tunkelang

Web Science: How is it different? Daniel Tunkelang, LinkedIn Keynote Address at ACM Web Science 2014 Conference The scientific method of observation, measurement, and experiment may be our greatest achievement as a species. The technological innovation we enjoy today is the product of a culture of systematized scientific experimentation. But historically scientific experimentation has been expensive. Experiments consumed natural resources, took a long time to conduct, and required even more time and labor to analyze. In order to be productive, scientists have had to factor these costs into their work and to optimize accordingly. Web science is different. Not, as some have speciously argued, because big data has made the scientific method obsolete. The key difference is that web science has changed the economics of scientific experimentation. Thus, even as web scientists apply the traditional scientific method, they optimize based on very different economics. In this talk, I'll survey how web science has changed our approach to experimentation, for better and for worse. Specifically, I'll talk about differences in hypothesis generation, offline analysis, and online testing. Bio Daniel Tunkelang is Head of Query Understanding at LinkedIn, where he previously formed and led the product data science team. LinkedIn search allows members to find people, companies, jobs, groups and other content. His team aims to provide users with the best possible results that satisfy their information needs and help to get insights from professional data. Tunkelang has BS and MS degrees in computer science and math from MIT, and a PhD in computer science from CMU. He co-founded the annual symposium on human-computer interaction and information retrieval (HCIR) and wrote the first book on Faceted Search (Morgan and Claypool 2009). Prior to joining LinkedIn, Tunkelang was Chief Scientist of Endeca (acquired by Oracle in 2011 for $1.1B) and leader of the local search quality team at Google, mapping local businesses to their home pages. He is the co-inventor of 20 patents.

Web science - How is it different?

Daniel Tunkelang

Keynote at CIKM 2013 Workshop on Data-driven User Behavioral Modelling and Mining from Social Media Social Search in a Professional Context Daniel Tunkelang (LinkedIn) Social networks bring a new dimension to search. Instead of looking for web pages or text documents, LinkedIn members search a world of entities connected by a rich graph of relationships. Search is a fundamental part of the LinkedIn ecosystem, as it helps our members find and be found. Unlike most search applications, LinkedIn's search experience is highly personalized: two LinkedIn members performing the same search query are likely to see completely different results. Delivering the right results to the right person depends on our ability to leverage our each member's unique professional identity and network. In this talk, I'll describe the kinds of search behavior we see on LinkedIn, and some of the approaches we've taken to help our members address their information needs.

Social Search in a Professional Context

Daniel Tunkelang

Search as Communication: Lessons from a Personal Journey by Daniel Tunkelang (Head of Query Understanding, LinkedIn) Presented at Etsy's Code as Craft Series on May 21, 2013 When I tell people I spent a decade studying computer science at MIT and CMU, most assume that I focused my studies in information retrieval — after all, I’ve spent most of my professional life working on search. But that’s not how it happened. I learned about information extraction as a summer intern at IBM Research, where I worked on visual query reformulation. I learned how search engines work by building one at Endeca. It was only after I’d hacked my way through the problem for a few years that I started to catch up on the rich scholarly literature of the past few decades. As a result, I developed a point of view about search without the benefit of academic conventional wisdom. Specifically, I came to see search not so much as a ranking problem as a communication problem. In this talk, I’ll explain my communication-centric view of search, offering examples, general techniques, and open problems. -- Daniel Tunkelang is Head of Query Understanding at LinkedIn. Educated at MIT and CMU, he has his career working on big data, addressing key challenges in search, data mining, user interfaces, and network analysis. He co-founded enterprise search and business intelligence pioneer Endeca, where he spent a decade as its Chief Scientist. In 2011, Endeca was acquired by Oracle for over $1B. Previous to LinkedIn, he led a team at Google working on local search quality. Daniel has authored fifteen patents, written a textbook on faceted search, and created the annual symposium on human-computer interaction and information retrieval.

Search as Communication: Lessons from a Personal Journey

Daniel Tunkelang

Enterprise Search: How Do We Get There From Here? by Daniel Tunkelang (Head of Query Understanding, LinkedIn) Keynote at 2013 Enterprise Search Summit We've been tackling the challenges of enterprise and site search for at least 3 decades. We've succeeded to the point that search is the gateway to many of our information repositories. Nonetheless, users of enterprise search systems are frustrated with these systems' shortcomings. We see this frustration in surveys, but, more importantly, most of us experience it personally in our daily work life. We all dream of a world where searching any information repository is as effective as searching the web—perhaps even more so. A world where we find what we're looking for, or quickly determine that it doesn't exist. Is this Utopia possible? If so, how do we get there from here? Or at least somewhere close? In this talk, Tunkelang reviews the track record of enterprise search. He talks about what's worked and what hasn't, especially as compared to web search. Finally, he proposes some paths to bring us closer to our dream. -- Daniel Tunkelang is Head of Query Understanding at LinkedIn. Educated at MIT and CMU, he has his career working on big data, addressing key challenges in search, data mining, user interfaces, and network analysis. He co-founded enterprise search and business intelligence pioneer Endeca, where he spent a decade as its Chief Scientist. In 2011, Endeca was acquired by Oracle for over $1B. Previous to LinkedIn, he led a team at Google working on local search quality. Daniel has authored fifteen patents, written a textbook on faceted search, and created the annual symposium on human-computer interaction and information retrieval.

Enterprise Search: How do we get there from here?

Daniel Tunkelang

Big Data, We Have a Communication Problem by Daniel Tunkelang Presented on April 30, 2013 at the TTI/Vanguard Conference on Ginormous Systems http://www.ttivanguard.com/conference/2013/ginormous.html It's a cliché that we live in a world of Big Data. But the bottleneck in understanding data is not computational. Rather, the biggest challenge is designing technical solutions that effectively leverage human cognitive ability. Data analysis systems should augment people's capabilities rather than replace them. This argument is as old as computer science itself: in 1962, Doug Engelbart said that the goal of technology is “the enhancement of human intellect by increasing the capability of a human to approach a complex problem situation.” Algorithms extract signal from raw data, but people fill in the gaps, creating models and evaluating analyses. Empowering people to understand data is not just a surface problem of building better interfaces and visualizations. We need to interact with data not only after performing computational analysis, but throughout the analysis process in order to improve our models and algorithms. In order to do so, we need tools and processes specifically designed to offer people transparency, guidance, and control. Human-computer information retrieval has been revolutionizing our approach to information seeking -- no modern search engine limits users to black-box relevance ranking and ten blue links. We need to take similar steps in our analysis of big data, making people the center of the analysis process and developing the technical innovations that enable people to fulfill this role.

Big Data, We Have a Communication Problem

Daniel Tunkelang

How To Interview a Data Scientist Daniel Tunkelang Presented at the O'Reilly Strata 2013 Conference Video: https://www.youtube.com/watch?v=gUTuESHKbXI Interviewing data scientists is hard. The tech press sporadically publishes “best” interview questions that are cringe-worthy. At LinkedIn, we put a heavy emphasis on the ability to think through the problems we work on. For example, if someone claims expertise in machine learning, we ask them to apply it to one of our recommendation problems. And, when we test coding and algorithmic problem solving, we do it with real problems that we’ve faced in the course of our day jobs. In general, we try as hard as possible to make the interview process representative of actual work. In this session, I’ll offer general principles and concrete examples of how to interview data scientists. I’ll also touch on the challenges of sourcing and closing top candidates.

How to Interview a Data Scientist

Daniel Tunkelang

Information, Attention, and Trust: A Hierarchy of Needs

Daniel Tunkelang

Data By The People, For The People Daniel Tunkelang Director, Data Science at LinkedIn Invited Talk at the 21st ACM International Conference on Information and Knowledge Management (CIKM 2012) LinkedIn has a unique data collection: the 175M+ members who use LinkedIn are also the content those same members access using our information retrieval products. LinkedIn members performed over 4 billion professionally-oriented searches in 2011, most of those to find and discover other people. Every LinkedIn search and recommendation is deeply personalized, reflecting the user's current employment, career history, and professional network. In this talk, I will describe some of the challenges and opportunities that arise from working with this unique corpus. I will discuss work we are doing in the areas of relevance, recommendation, and reputation, as well as the ecosystem we have developed to incent people to provide the high-quality semi-structured profiles that make LinkedIn so useful. Bio: Daniel Tunkelang leads the data science team at LinkedIn, which analyzes terabytes of data to produce products and insights that serve LinkedIn's members. Prior to LinkedIn, Daniel led a local search quality team at Google. Daniel was a founding employee of faceted search pioneer Endeca (recently acquired by Oracle), where he spent ten years as Chief Scientist. He has authored fourteen patents, written a textbook on faceted search, created the annual workshop on human-computer interaction and information retrieval (HCIR), and participated in the premier research conferences on information retrieval, knowledge management, databases, and data mining (SIGIR, CIKM, SIGMOD, SIAM Data Mining). Daniel holds a PhD in Computer Science from CMU, as well as BS and MS degrees from MIT.

Data By The People, For The People

Daniel Tunkelang

Content, Connections, and Context Daniel Tunkelang, LinkedIn Keynote at Workshop on Recommender Systems and the Social Web At 6th ACM International Conference on Recommender Systems (RecSys 2012) Recommender systems for the social web combine three kinds of signals to relate the subject and object of recommendations: content, connections, and context. Content comes first - we need to understand what we are recommending and to whom we are recommending it in order to decide whether the recommendation is relevant. Connections supply a social dimension, both as inputs to improve relevance and as social proof to explain the recommendations. Finally, context determines where and when a recommendation is appropriate. I'll talk about how we use these three kinds of signals in LinkedIn's recommender systems, as well as the challenges we see in delivering social recommendations and measuring their relevance.

Content, Connections, and Context

Daniel Tunkelang

Keynote at 2012 Semantic Technology and Business Conference Scale, Structure, and Semantics Daniel Tunkelang, LinkedIn Science fiction has a mixed track record when it comes to anticipating technological innovations. While Jules Verne fared well with with his predictions of submarine and space technology, artificial intelligence hasn't produced anything like Arthur C. Clarke's HAL 9000. Instead, we've managed to elicit intelligence from machines through unexpected means. Search engines have achieved remarkable success in organizing the world's information by crawling the web, indexing documents, and exploiting link structure to establish authoritativeness. At LinkedIn, we apply large-scale analytics to terabytes of semistructured data to deliver products and insights that serve our 150M+ members. Semantics emerge when we apply the right analytical techniques to a sufficient quality and quantity of data. In this talk, I will describe how LinkedIn's huge and rich graph of relationship data that powers the products our users love. I believe that the lessons we have learned apply broadly to other semantic applications. While quantity and quality of data are the key challenges to delivering a semantically rich experience, the key is to create the right ecosystem that incents people to give you good data, which then forms the basis for great data products.

Scale, Structure, and Semantics

Daniel Tunkelang

Presentation from O'Reilly Strata 2012 on Big Data Humans, Machines, and the Dimensions of Microwork Daniel Tunkelang (LinkedIn) Claire Hunsaker (Samasource) The advent of crowdsourcing has wildly expanded the ways we think of incorporating human judgments into computational workflows. Computer scientists, economists, and sociologists have explored how to effectively and efficiently distribute microwork tasks to crowds and use their work as inputs to create or improve data products. Simultaneously, crowdsourcing providers are exploring the bounds of mechanical QA flows, worker interfaces, and workforce management systems. But what tasks should be performed by humans rather than algorithms? And what makes a set of human judgments robust? Quantity? Consensus? Quality or trustworthiness of the workers? Moreover, the robustness of judgments depends not only on the workers, but on the task design. Effective crowdsourcing is a cooperative endeavor. In this talk, we will analyze various dimensions of microwork that characterize applications, tasks, and crowds. Drawing on our experience at companies that have pioneered the use of microwork (Samasource) and data science (LinkedIn), we will offer practical advice to help you design crowdsourcing workflows to meet your data product needs.

Strata 2012: Humans, Machines, and the Dimensions of Microwork

Daniel Tunkelang

Mehr von Daniel Tunkelang (20)

Query Understanding and Ecommerce

Semantic Equivalence of e-Commerce Queries

Helping Searchers Satisfice through Query Understanding

MMM, Search!

Enterprise Intelligence

Query Understanding: A Manifesto

Where should you put your data scientists?

Data Science: A Mindset for Productivity

My Three Ex’s: A Data Science Approach for Applied Machine Learning

Web science - How is it different?

Social Search in a Professional Context

Search as Communication: Lessons from a Personal Journey

Enterprise Search: How do we get there from here?

Big Data, We Have a Communication Problem

How to Interview a Data Scientist

Information, Attention, and Trust: A Hierarchy of Needs

Data By The People, For The People

Content, Connections, and Context

Scale, Structure, and Semantics

Strata 2012: Humans, Machines, and the Dimensions of Microwork

Kürzlich hochgeladen

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Scaling API-first – The story of a global engineering organization

Radu Cotescu

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Histor y of HAM Radio presentation slide

vu2urc

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

Kürzlich hochgeladen (20)

The 7 Things I Know About Cyber Security After 25 Years | April 2024

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Boost Fertility New Invention Ups Success Rates.pdf

Data Cloud, More than a CDP by Matt Robison

Automating Google Workspace (GWS) & more with Apps Script

Exploring the Future Potential of AI-Enabled Smartphone Processors

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Scaling API-first – The story of a global engineering organization

How to Troubleshoot Apps for the Modern Connected Worker

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

GenAI Risks & Security Meetup 01052024.pdf

Partners Life - Insurer Innovation Award 2024

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

presentation ICT roal in 21st century education

Strategies for Landing an Oracle DBA Job as a Fresher

Histor y of HAM Radio presentation slide

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Better Search Through Query Understanding

1. Recruiting SolutionsRecruiting SolutionsRecruiting Solutions Daniel Tunkelang Head, Query Understanding better search through query understanding

2. overview  query understanding: what is it?  how we do query understanding at LinkedIn  some other thoughts from search in the wild what I’m not going to cover: 2

3. Information need query select from results rank using IR model user: system: tf-idf PageRank bird’s-eye view of how a search engine works 3

4. Information need query select from results rank using IR model user: system: tf-idf PageRank query understanding 4

5. search is a communication problem 5

6. 6 tag: skill OR title related skills: search, ranking, … tag: company id: 1337 industry: internet verticals: people, jobs intent: exploratory

7. query understanding pipeline 7 spellcheck query tagging vertical intent prediction query expansion raw query structured query + annotations

8. query understanding pipeline 8 spellcheck query tagging vertical intent prediction query expansion raw query structured query + annotations

9. 9 fix obvious typos help users spell names spelling correction

10. spelling out the details 10 PEOPLE NAMES COMPANIES TITLES PAST QUERIES n-grams marissa => ma ar ri is ss sa metaphone mark/marc => MRK co-occurrence counts marissa:mayer = 1000 marisa meyer yahoo marissa marisa meyer mayer yahoo

11. spelling out the details 11 problem: corpus as well as query logs contain many spelling errors certain spelling errors are quite frequent while genuine words (especially names) might be infrequent

12. spelling out the details 12 problem: corpus & query logs contain spelling errors solution: use query chains to infer correct spelling [product manger] [product manager] CLICK [marissa mayer] CLICK

13. query understanding pipeline 13 spellcheck query tagging vertical intent prediction query expansion raw query structured query + annotations

14. query tagging: identifying entities in the query 14 TITLE CO GEO TITLE-237 software engineer software developer programmer … CO-1441 Google Inc. Industry: Internet GEO-7583 Country: US Lat: 42.3482 N Long: 75.1890 W (RECOGNIZED TAGS: NAME, TITLE, COMPANY, SCHOOL, GEO, SKILL )

15. query tagging: identifying entities in the query 15 TITLE CO GEO MORE PRECISE MATCHING WITH DOCUMENTS

16. entity-based filtering 16 BEFORE

17. entity-based filtering 17 AFTER BEFORE

18. entity-based filtering 18 BEFORE

19. entity-based filtering 19 AFTER BEFORE

20. entity-based suggestions 20

21. entity-based suggestions 21

22. query tagging: sequential model 22 EMISSION PROBABILITIES (learned from user profiles) TRANSITION PROBABILITIES (learned from query logs) TRAINING

23. query tagging: sequential model 23 INFERENCE given a query, find the most likely sequence of tags

24. query understanding pipeline 24 spellcheck query tagging vertical intent prediction query expansion raw query structured query + annotations

25. vertical intent prediction: distribution 25 JOBS PEOPLE COMPANIES (probability distribution over verticals)

26. vertical intent prediction: relevance 26 [company] [employees] [jobs] [name search]

27. query understanding pipeline 27 spellcheck query tagging vertical intent prediction query expansion raw query structured query + annotations

28. 28 query expansion: name synonyms

29. 29 query expansion: job title synonyms

30. 30 query expansion: signals [jon] [jonathan] CLICK trained using query chains: [programmer] [developer] CLICK symmetric but not transitive! [francis] ⇔ [frank] [franklin] ⇔ [frank] [francis] ≠ [franklin] [software engineer] [software developer] CLICK context based! [software engineer] => [software developer] [civil engineer] ≠ [civil developer]

31. query understanding pipeline 31 spellcheck query tagging vertical intent prediction query expansion raw query structured query + annotations

32. 32 what else can we learn from search in the wild?

33. don’t guess when it’s better to ask 33 vs.

34. clarify then refine 34 computers books

35. give users transparency, guidance, and control 35

36. think beyond individual search queries 36 Gene Golovchinsky, FXPAL

37. know when you don’t know 37 Claudia Hauff, Query Difficulty for Digital Libraries [2009]

38. 38 Daniel Tunkelang dtunkelang@linkedin.com https://linkedin.com/in/dtunkelang

Better Search Through Query Understanding

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Better Search Through Query Understanding

Ähnlich wie Better Search Through Query Understanding (20)

Mehr von Daniel Tunkelang

Mehr von Daniel Tunkelang (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Better Search Through Query Understanding