Presentation at Web Directions South, 2011

1. Making the Web Searchable Peter Mika Senior Researcher and Data Architect Yahoo! Inc.

3. Convergence of Search and Online Media

4. It used to be pretty simple…

5. Yahoo! today is a global network of online media sites

6. ... with search as an important entry point to content Information box with content from and links to Yahoo! Travel Points of interest in Vienna, Austria Since Aug, 2010, ‘regular’ search results are ‘Powered by Bing’ Shopping results from Yahoo! Shopping

7. Conversely, online media as an entry point to search Hovering over an underlined phrase triggers a search for related news items.

8. Aggregation across space: hyperlocal pages Hyperlocal: showing content from across Yahoo that is relevant to a particular neighbourhood.

9. Aggregation across entity types: special events

10. Personalization Yahoo’s Content Optimization Relevance Engine (CORE) technology uses machine learning to predict click behavior based on user profile Display advertizing is also personalized by default. Users can opt-out of behavioral targeting through AdChoices.

11. Contextualization Show related content Social discovery: connect with friends watching the same

13. Semantic technologies for Search

14. Search is really fast, without necessarily being intelligent

16. Not just search…

17. What it’s like to be a machine? Roi Blanco

18. What it’s like to be a machine?  ✜ Θ ♬♬ţğ   ✜ Θ ♬♬ţğ √∞  ®ÇĤĪ ✜★  ♬☐ ✓✓ ţğ  ★  ✜   ✪✚✜ Δ ΤΟŨŸÏĞÊϖυτρ℠≠⅛⌫ Γ ≠ =⅚ ©§ ★✓♪ ΒΓΕ  ℠   ✖ Γ ♫⅜±  ⏎ ↵⏏  ☐ģğğğμλκσςτ   ⏎  ⌥ °¶§ΥΦΦΦ ✗✕ ☐ 

24. Example: the hCard microformat <cite class=&quot;vcard&quot; > <a class=&quot;fn url&quot; rel=&quot;friend colleague met” href=&quot;http://meyerweb.com/&quot;> Eric Meyer</a> </cite> wrote a post (<cite> <a href=&quot;http://meyerweb.com/eric/thoughts/2005/12/16/tax-relief/&quot;> Tax Relief</a></cite>) about an unintentionally humorous letter he received from the <span class=&quot;vcard” > <a class=&quot;fn org url&quot; href=&quot;http://irs.gov/&quot;> Internal Revenue Service</a> </span>. <div class=&quot;vcard&quot; > <a class=&quot;email fn&quot; href=&quot;mailto:jfriday@host.com&quot;>Joe Friday</a> <div class=&quot;tel&quot; >+1-919-555-7878</div> <div class=&quot;title&quot; >Area Administrator, Assistant</div> </div>

38. RDFa on the rise Percentage of URLs with embedded metadata in various formats 510% increase between March, 2009 and October, 2010

40. Semantic technologies for Data Integration

41. Today’s world is a Web of Pages

42. All these pages come from structured knowledge about people, places, and things MLB team Chicago Cubs Is a Chicago Barack Obama Carlos Zambrano 10% off tickets for plays for plays in from

43. This underlying world is WOO—the Web of Objects MLB team Chicago Cubs Is a Chicago Barack Obama Carlos Zambrano 10% off tickets for plays for plays in from

44. Today our knowledge of this world is siloed, incomplete, inconsistent, inaccurate, and hard to reuse Sports Entertainment Finance Local Shopping Upcoming MLB team Chicago Cubs isa Chicago Scott Roy Carlos Zambrano 10% off tickets for plays for plays in from

45. Our vision is a single shared knowledge base—accurate, scalable, and easy to reuse MLB team Chicago Cubs isa Chicago Barack Obama Carlos Zambrano 10% off tickets for plays for plays in from

46. Knowledge comes from many sources Entities Attributes Show times and other information for US movies from source B Harry Potter and the Deathly Hallows part II Show times Show times for Harry Potter and the Deathly Hallows part II

47. Combining these requires working with complementary, parallel, and overlapping sources Attributes Entities Cast information for global movies from Wikipedia Cast information for US movies from source A Cast and show time information for global movies from licensed feeds

48. There is a tremendous opportunity to do this directly from Web pages, reverse engineering the Web Attributes Entities Information from structured data extraction on billions of Web pages

53. Value #1 — Breadth, depth, and accuracy at scale Real entities Dups, errors, and outdated entities Up-to-date correct entities Incorrect store URL No photo We show many entities we shouldn’t No business hours WOO improves our breadth, depth, and accuracy by combining knowledge from alternative sources, and by modernizing how we do matching, blending, and de-duping

54. Value #2 — Agility launching new experiences Answers instead of links WOO lets us quickly create entity centric DD modules using the existing knowledge in the KB Related knowledge in context The integrated KB lets us show relevant knowledge from one Yahoo property on other properties and off network Emerging markets and tail pages The KB gets us deep into the tail by combining and blending knowledge from many sources

56. Innovative media companies are moving in this direction Courtesy of Silver Oliver (BBC)

57. Innovative media companies are moving in this direction Courtesy of Evan Sandhaus (NYT).