SlideShare ist ein Scribd-Unternehmen logo
1 von 52
Downloaden Sie, um offline zu lesen
An Optimization Framework for
   Query Recommendation
   Aris Anagnostopoulos, Luca Becchetti,

      Carlos Castillo, Aristides Gionis
Source: B. Precourt at UWM.
An Optimization Framework for Query Recommendation
An Optimization Framework for Query Recommendation
As you set

out for Ithaca,

hope your road

is a long one,

full of adventure,

full of discovery.

          K. Kavafis
Problem

Given a set of possible user histories




6    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Problem

Given a value for different states




7    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Problem

Given a value for different states




8    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Problem

“Nudge” the users in a certain direction

                                -
     -

                           +

         +




9    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Problem

“Nudge” the users in a certain direction




10   A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Problem definition


     Given a set of possible user sessions

     Given a certain value for different states

     “Nudge” the users in a certain direction

                                  Objectives
      – 1. collect a large reward along the way -or-
      – 2. end the session at a rewarding action

11       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Constraints

•    We are not almighty




                                                                  Source: not-of-this-earth.com


12    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Constraints

•     We are not almighty
     – We can only suggest, not order




13      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Constraints

•     We are not almighty
     – We can only suggest, not order

•     We are not all-knowing




                                                                    Source: Wikimedia commons.


14      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Constraints

•     We are not almighty
     – We can only suggest, not order

•     We are not all-knowing
     – We do not know how the users will react




15      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Framework
Setting

•     Talk: query recommendation
•     Paper: general framework
     – E.g.: optimizing links in web-sites




17      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
An Optimization Framework for Query Recommendation
An Optimization Framework for Query Recommendation
Query recommendations

•     Reformulation probabilities
        – P(q,q') original


                                          0.3           central park map

                                          0.2           central park hotel
     central park
                                          0.2           central park new york
                                          0.3
                                                        central park zoo



20      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Query recommendations

•     Reformulation probabilities
        – P(q,q') original
        – P'(q,q') perturbed = P(q,q') + ρ(Q,q,q')

                                    -0.1 0.3            central park map

                                    -0.1 0.2            central park hotel
     central park
                                   +0.1 0.2             central park new york
                                   +0.1 0.3
                                                        central park zoo



21      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Query recommendations

•     Reformulation probabilities
        – P(q,q') original
        – P'(q,q') perturbed = P(q,q') + ρ(Q,q,q')

                                          0.2           central park map

                                          0.1           central park hotel
     central park
                                          0.3           central park new york
                                          0.4
                                                        central park zoo



22      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Example values w(·)

• Search engine results page
     – Quality of search results

• General page
     – Dwell time
     – User ratings




23   A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Objective functions U(·)

            Kavafian                                       Machiavellian
     “a road full of adventure”                          “ends justify means”



      U(<q1,q2,...,qt>) = Σw(qi)                         U(<q1,q2,...,qt>) = w(qt)




24         A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
The Kavafian objective

Useful when users want to explore, or be entertained




25    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
The Kavafian objective

Useful when users want to explore, or be entertained

                                                                 +         “funny video”

                                                                     -




26    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Optimization problem

•    Given:
      – Original transition probabilities P
      – Starting node q
      – Node values w
      – Perturbation function ρ

•    Add up to k links (per node), maximize
     expected utility of paths starting at q

27    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Single-step recommendation

• Recommend now, leave user alone later



     q                                                                        t




28       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Single-step recommendation

• Recommend now, leave user alone later

                   +


     q                                                                        t


                   -




29       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Multi-step recommendation

• Recommend at each step in the future



     q                                                                        t




30       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Multi-step recommendation

• Recommend at each step in the future
                                               -

                   -                                                       +

                                               +
     q                            -                        +                   t
                                              -

                   +                                                   -

                                                  +




31       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Multi-step case

•     Multi- and Single-step
      recommendation are NP-complete
     – Reduction from MAXIMUM-COVER

•     Heuristic for multi-step problem:
       – at each node, assume rest of the graph is
           unperturbed when computing utility of
           adding an edge


32     A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Single-step case

•    Greedy heuristic for “Machiavellian”
     objective: find (qi, qj) maximizing

                    ρij(E[U(path(qj))] – wi)

•    Repeat k times




33    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Observation

• Greedy heuristic achieves utility at least (1-x) of
   the optimum, with x << 1 in cases of practical
   interest
     – It is possible to construct pathological
         instances s.t. that greedy performs poorly
     – x depends on termination probability at qi
         and probabilities of following
         recommendations


34     A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Application
Large-scale experiment

 We observe perturbed probabilities P'(q,q'),
unless we disable search assist to see P(q,q')




36    A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Empirical observations

• Notation:
     – τq, τ'q session-end probabilities

• Recommendations decrease termination
   probability:
     •     Average τq≈0.90
     •     Average τ'q≈0.84


37       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Termination probability
Empirical observations

•    Recommendations decrease
     termination probability, τq≈0.90 τ'q≈0.84

•    Decrease is almost entirely due to
     more clicks on recommendations




39     A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
An Optimization Framework for Query Recommendation
Empirical observations

•    Recommendations decrease
     termination probability from ≈0.90 to
     ≈0.84
•    Decrease is almost entirely due to
     more clicks on recommendations
•    ρ is difficult to estimate



41     A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
P'(q,q'): with recommendations




                                 P(q,q'): without recommendations
P'(q,q'): with recommendations




                                                 There is an increase


                                 P(q,q'): without recommendations
Many un-seen suggestions
P'(q,q'): with recommendations   are clicked when shown




                                 P(q,q'): without recommendations
The rest are in general
P'(q,q'): with recommendations                 difficult to predict




                                 P(q,q'): without recommendations
So what do we do?

•     We approximate ρ by a linear function on
     – P(q,q')
     – Textual similarity of q and q'
     – Terminal probability of q

•     We have a low accuracy on this prediction
     – r ≈ 0.5
•     We use as weights the CTR on results


46       A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Evaluation
Evaluation results


     Baselines:
        –    Prefer queries with large w
        –    Prefer queries with large ρ
        –    Prefer queries with large ρw

     Greedy heuristic performs better for both
     utility functions


48      A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Expected utility




“Kavafian” objective   “Machiavellian” objective
Evaluation results


     Greedy heuristic performs well

     What about relevance?
     
         420 queries assessed by 3 judges
     
         There were no significant changes in
         relevance between systems




50        A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Conclusions

• General framework for
  “nudging” users in a
  certain direction

• Open algorithmic and
  practical questions
• In the paper: related work




51     A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis   WSDM'10
Q&A

Weitere ähnliche Inhalte

Kürzlich hochgeladen

IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideHironori Washizaki
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Juan Carlos Gonzalez
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...Daniel Zivkovic
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementNuwan Dias
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 

Kürzlich hochgeladen (20)

IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK GuideIEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
IEEE Computer Society’s Strategic Activities and Products including SWEBOK Guide
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?Governance in SharePoint Premium:What's in the box?
Governance in SharePoint Premium:What's in the box?
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
The Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API ManagementThe Kubernetes Gateway API and its role in Cloud Native API Management
The Kubernetes Gateway API and its role in Cloud Native API Management
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 

Empfohlen

AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfmarketingartwork
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsKurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summarySpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best PracticesVit Horky
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project managementMindGenius
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at WorkGetSmarter
 

Empfohlen (20)

AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity -  Best PracticesTime Management & Productivity -  Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...
 
12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work12 Ways to Increase Your Influence at Work
12 Ways to Increase Your Influence at Work
 
ChatGPT webinar slides
ChatGPT webinar slidesChatGPT webinar slides
ChatGPT webinar slides
 
More than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike RoutesMore than Just Lines on a Map: Best Practices for U.S Bike Routes
More than Just Lines on a Map: Best Practices for U.S Bike Routes
 

An Optimization Framework for Query Recommendation

  • 1. An Optimization Framework for Query Recommendation Aris Anagnostopoulos, Luca Becchetti, Carlos Castillo, Aristides Gionis
  • 5. As you set out for Ithaca, hope your road is a long one, full of adventure, full of discovery. K. Kavafis
  • 6. Problem Given a set of possible user histories 6 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 7. Problem Given a value for different states 7 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 8. Problem Given a value for different states 8 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 9. Problem “Nudge” the users in a certain direction - - + + 9 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 10. Problem “Nudge” the users in a certain direction 10 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 11. Problem definition  Given a set of possible user sessions  Given a certain value for different states  “Nudge” the users in a certain direction Objectives – 1. collect a large reward along the way -or- – 2. end the session at a rewarding action 11 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 12. Constraints • We are not almighty Source: not-of-this-earth.com 12 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 13. Constraints • We are not almighty – We can only suggest, not order 13 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 14. Constraints • We are not almighty – We can only suggest, not order • We are not all-knowing Source: Wikimedia commons. 14 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 15. Constraints • We are not almighty – We can only suggest, not order • We are not all-knowing – We do not know how the users will react 15 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 17. Setting • Talk: query recommendation • Paper: general framework – E.g.: optimizing links in web-sites 17 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 20. Query recommendations • Reformulation probabilities – P(q,q') original 0.3 central park map 0.2 central park hotel central park 0.2 central park new york 0.3 central park zoo 20 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 21. Query recommendations • Reformulation probabilities – P(q,q') original – P'(q,q') perturbed = P(q,q') + ρ(Q,q,q') -0.1 0.3 central park map -0.1 0.2 central park hotel central park +0.1 0.2 central park new york +0.1 0.3 central park zoo 21 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 22. Query recommendations • Reformulation probabilities – P(q,q') original – P'(q,q') perturbed = P(q,q') + ρ(Q,q,q') 0.2 central park map 0.1 central park hotel central park 0.3 central park new york 0.4 central park zoo 22 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 23. Example values w(·) • Search engine results page – Quality of search results • General page – Dwell time – User ratings 23 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 24. Objective functions U(·) Kavafian Machiavellian “a road full of adventure” “ends justify means” U(<q1,q2,...,qt>) = Σw(qi) U(<q1,q2,...,qt>) = w(qt) 24 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 25. The Kavafian objective Useful when users want to explore, or be entertained 25 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 26. The Kavafian objective Useful when users want to explore, or be entertained + “funny video” - 26 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 27. Optimization problem • Given: – Original transition probabilities P – Starting node q – Node values w – Perturbation function ρ • Add up to k links (per node), maximize expected utility of paths starting at q 27 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 28. Single-step recommendation • Recommend now, leave user alone later q t 28 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 29. Single-step recommendation • Recommend now, leave user alone later + q t - 29 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 30. Multi-step recommendation • Recommend at each step in the future q t 30 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 31. Multi-step recommendation • Recommend at each step in the future - - + + q - + t - + - + 31 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 32. Multi-step case • Multi- and Single-step recommendation are NP-complete – Reduction from MAXIMUM-COVER • Heuristic for multi-step problem: – at each node, assume rest of the graph is unperturbed when computing utility of adding an edge 32 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 33. Single-step case • Greedy heuristic for “Machiavellian” objective: find (qi, qj) maximizing ρij(E[U(path(qj))] – wi) • Repeat k times 33 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 34. Observation • Greedy heuristic achieves utility at least (1-x) of the optimum, with x << 1 in cases of practical interest – It is possible to construct pathological instances s.t. that greedy performs poorly – x depends on termination probability at qi and probabilities of following recommendations 34 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 36. Large-scale experiment We observe perturbed probabilities P'(q,q'), unless we disable search assist to see P(q,q') 36 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 37. Empirical observations • Notation: – τq, τ'q session-end probabilities • Recommendations decrease termination probability: • Average τq≈0.90 • Average τ'q≈0.84 37 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 39. Empirical observations • Recommendations decrease termination probability, τq≈0.90 τ'q≈0.84 • Decrease is almost entirely due to more clicks on recommendations 39 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 41. Empirical observations • Recommendations decrease termination probability from ≈0.90 to ≈0.84 • Decrease is almost entirely due to more clicks on recommendations • ρ is difficult to estimate 41 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 42. P'(q,q'): with recommendations P(q,q'): without recommendations
  • 43. P'(q,q'): with recommendations There is an increase P(q,q'): without recommendations
  • 44. Many un-seen suggestions P'(q,q'): with recommendations are clicked when shown P(q,q'): without recommendations
  • 45. The rest are in general P'(q,q'): with recommendations difficult to predict P(q,q'): without recommendations
  • 46. So what do we do? • We approximate ρ by a linear function on – P(q,q') – Textual similarity of q and q' – Terminal probability of q • We have a low accuracy on this prediction – r ≈ 0.5 • We use as weights the CTR on results 46 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 48. Evaluation results  Baselines: – Prefer queries with large w – Prefer queries with large ρ – Prefer queries with large ρw  Greedy heuristic performs better for both utility functions 48 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 49. Expected utility “Kavafian” objective “Machiavellian” objective
  • 50. Evaluation results  Greedy heuristic performs well  What about relevance?  420 queries assessed by 3 judges  There were no significant changes in relevance between systems 50 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 51. Conclusions • General framework for “nudging” users in a certain direction • Open algorithmic and practical questions • In the paper: related work 51 A. Anagnostopoulos, L. Becchetti, C. Castillo, A. Gionis WSDM'10
  • 52. Q&A