Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Making search better by tracking & 
utilizing user search behavior 
Sameer Maggon
Agenda 
• About Me 
! 
• Measuring Search Quality 
! 
• Quality Metrics based on users interactions 
! 
• Quality Metrics ...
Sameer Maggon 
Founder of Cloud based Zero Management Solution 
for Search 
Built relevance based search platform for AT&T...
Search seems Easy 
1 2 3 
Index all content Put a search box Show google 
4 
like results
Is It? 
• How do you know that your users are finding what they 
5 
are looking for? 
! 
• How do you know what impact you...
How do we measure search? 
Usage Data Editorial Labeling 
6 
Collect, Analyze & Report on 
interactions users are having 
...
Usage Data: Key Metrics to look at 
7 
• No Result Search % 
• Search Exits % 
• CTR % 
! 
• Average Click Position 
• MRR...
Search Analytics: Examples 
8
9 
Editorial Labeling 
• Precision Recall 
! 
• DCG (Discounted 
Cumulative Gain) 
! 
• nDCG (Normalized DCG) 
http://en.w...
Improving Search Results 
Attack Low Hanging fruit first 
• Popular No Result Searches - Can we use keyword stuffing? 
• P...
Using Popularity to affect search ranking 
DEMO 
11
www.measuredsearch.com 
sameer@measuredsearch.com
Nächste SlideShare
Wird geladen in …5
×

Making search better by tracking & utilizing user search behavior

Improving search experience is a very data driven & iterative process. Measuring the quality of search results and having the tools to measure, make sense and then improve your search ranking algorithms is key to the entire process. We'll talk through some aspects of how to measure quality for given search results and key metrics that the teams need to track to as they make changes to their search to make it better. Later during the talk, we’ll show a very basic example of how one can start to incorporate user behavior as part of the search results to influence the ranking of the results.

  • Als Erste(r) kommentieren

Making search better by tracking & utilizing user search behavior

  1. 1. Making search better by tracking & utilizing user search behavior Sameer Maggon
  2. 2. Agenda • About Me ! • Measuring Search Quality ! • Quality Metrics based on users interactions ! • Quality Metrics based on User Tagging (labelers) ! • Improving Search Results ! • Using user behavior to improve results ranking - Example 2
  3. 3. Sameer Maggon Founder of Cloud based Zero Management Solution for Search Built relevance based search platform for AT&T Interactive & properties including yp.com, buzz.com, yellowpages.com Consulted for numerous Startups to Fortune 500 companies around Search & Discovery. 3 Engineering Alumni @maggon http://linkedin.com/in/maggon
  4. 4. Search seems Easy 1 2 3 Index all content Put a search box Show google 4 like results
  5. 5. Is It? • How do you know that your users are finding what they 5 are looking for? ! • How do you know what impact your one-off fix has on an aggregate? ! • Seemingly good result list to one might be irrelevant to another (e.g. mosaic)
  6. 6. How do we measure search? Usage Data Editorial Labeling 6 Collect, Analyze & Report on interactions users are having with your search functionality. Get a set of users to mark top x results with “Relevant vs. Not” for a pre-determined sample set of searches. ! Then compute specific metrics based on those.
  7. 7. Usage Data: Key Metrics to look at 7 • No Result Search % • Search Exits % • CTR % ! • Average Click Position • MRR (Mean Reciprocal Rank) • Clicks per Search • Paging (how deep do people have to dig?) ! • Latency (Average, tp90 and tp95) Aggregate & Trends Trends Aggregate & Trends
  8. 8. Search Analytics: Examples 8
  9. 9. 9 Editorial Labeling • Precision Recall ! • DCG (Discounted Cumulative Gain) ! • nDCG (Normalized DCG) http://en.wikipedia.org/wiki/Discounted_cumulative_gain Relevant Not Relevant Relevant Not Relevant Not Relevant Relevant Not Relevant weight-age decreases as as you go down on an ordered list
  10. 10. Improving Search Results Attack Low Hanging fruit first • Popular No Result Searches - Can we use keyword stuffing? • Popular Search Exits - Eyeball outliers • Popular Searches with low CTRs • Generally improve Average Click Position / MRR via Utilizing Search Behavior to improve ranking • Utilize Popularity (click stream) to inform search ranking • Utilizing past search history to offer assistive features (search suggestions, related searches) 10 identifying patterns (impacts CTR) Advanced: Learning Models • Topic for some other time :)
  11. 11. Using Popularity to affect search ranking DEMO 11
  12. 12. www.measuredsearch.com sameer@measuredsearch.com

×