Overview of what's new in Enterprise Search for SharePoint 2010

1. Enterprise Search in SharePoint 2010 Bob GermanTechnology ArchitectMicrosoft Technology Center | Boston

2. Agenda SharePoint Search Options SharePoint Search Engine FAST for SharePoint Social Search Content Acquisition Federation

3. You Asked, We Listened! Scale – content and query scale becomes a non-issue User experience –guide users with rich, visual navigation Social and people search – search unlocks the value of the people and expertise Relevance –improving our matching and ranking capabilities and language support Platform –tools to customize, extend, or build new applications. Management & reliability –reliable crawling, scriptable admin, improved monitoring. Rich Content Processing–extract metadata from my enterprise content

5. Social search features and integration

6. SharePoint platform integration

7. End user and site administrator enablement

8. Common Connector Framework (BDC)

9. APIs and developer experience

10. Admin & deployment capabilities

12. SharePoint Search Engine What’s new in 2010?

13. SharePoint 2010 Shared Service Applications SSP Search UserProfiles Excel Calc UserProfiles Search HRBusinessData Visio Excel Calc BusinessData OfficeWeb Apps 3rd party Service IT Search http://itweb/ http://hrweb/ http://hrweb/ http://itweb/ Corp Farm Corp Farm SharePoint 2007 Model SharePoint 2010 Model

14. SharePoint Search SA Each Search Service Application (SSA): Administration component , database Query Topology Query Components (1+) Property Databases (1+) Crawl Topology Crawl Components (1+) Crawl Databases (1+) Services (Query, Admin)

15. Default Search Configuration Out of the Box: 1 Admin component 1 Crawl component 1 Query component Databases on drive specified in post-setup configuration Fully functional: Crawl content Develop customizations Query Crawl Admin Property DB Crawl DB Admin DB (other SharePoint DBs)

16. Component Scale-out Query Q2 Q1 Q2 Q1 Q6 Q5 Q4 Q3 Query: Segregate Partition for scale Passive mirrors provide failover Active mirrors increase throughput Crawl Additional crawl components Crawl Database Q2 Q1 Q1 Q2 Q5 Q6 Q3 Q4 Query Crawl Crawl Admin Crawl Crawl 100 million items! Property DB Crawl DB Crawl DB Admin DB Crawl DB Admin DB

17. Manageability – Monitor Crawl “Levels” of reports Current rate Crawl rate per type Crawl processing Filter by SSA, time Optimize crawl resources to achieve freshness goals.

18. Manageability – Monitor Query “Levels” of reports Current rate Overall Query Latency Detailed Backend Query Latency Optimize query latency Enable verbosequery latency, trend

19. FAST Search Engine High-end Search for SharePoint

20. Go Beyond the Search Box Visual, Conversational Search Sorting on any property Scrolling PowerPoint Previews Visual Best Bets Thumbnails Refinement with counts on any property Similarity Search

22. FAST

23. CCFSharePointAPI’s Document Processing Indexing Search Query &Result Administration FAST Search Engine Shared with SharePoint Search Engine

24. Content Pipeline Advanced Content Processing Out-of-the-Box Processing & refinement Configurable content processing pipeline Transform and process content automatically Strong linguistic coverage 80+ languages detected 45 languages with advanced linguistics features Property Extraction to create new metadata Prebuilt extractors: Person, Location, Company Generic extractors to extend Extensibility Location Company Company Company REDMOND, Wash., andOSLO, Norway — Jan. 8, 2008 Microsoft Corp. (Nasdaq “MSFT”) today announced that it will make an offer to acquire Fast Search & Transfer ASA (OSE: “FAST”), a leading provider of enterprise search solutions, through a cash tender offer for 19.00 Norwegian kroner (NOK) per share. This offer represents a 42 percent premium to the closing share price on Jan. 4, 2008 (the last trading day prior to this announcement), and values the fully diluted equity of FAST at 6.6 billion NOK (or approximately $1.2 billion U.S. ). FAST’s board of directors has unanimously recommended that its shareholders accept the offer. Mapper … Configurable Stages EntityExtraction Language Detection Format Conversion

28. Large Scale Deployment Add columns to extend document capacity Add rows to extend query capacity or fault tolerance Column 0 1 2 3 Indexer/Search Indexer/Search Indexer/Search Indexer/Search Row 0 Row 1 Indexer/Search Indexer/Search Indexer/Search Indexer/Search

29. Social Search

30. Go Behind the Search Box Connect with People and Expertise Vanity search Recently authored content Phonetic and nickname matching Improved result layout and hit-highlighting Sort by relevance, name or social distance Refine by query type, and many other pivots

31. Leveraging the Managed Metadata Service for better People Search Out of box: Responsibilities, Interest, Skills, Schools, are all taxonomy properties Automatic refinement on relevant pivots Extensibility Adding a new taxonomy property to the profile store

32. Click through improves relevance The wisdom of the masses helps the masses Documents are boosted by ‘popularity’: if many users find a document useful, it must be useful.

33. Search suggestions Helping users form better queries Pre-query suggestions Post-query suggestions

34. Social tags Recall Results with social tags are returned. Ranking Results with social tags are boosted. Refinement Users can filter results on social tags. Results Top social tags are displayed as metadata on each result.

35. Content Acquisition Feeding the Search Engines

36. Making Content Searchable Use Query Federation Query across external indexes Based on OpenSearch standard(http://www.opensearch.org/) Use Indexing Connectors Crawl, enumerate and create local index of content Use out-of-box connectors or write custom connector

37. Shared Out-of-box Indexing Connectors in SharePoint 2010 SharePoint content (Protocol Handler - PH) File shares (PH) Web sites (PH) People profiles (PH) Lotus Notes™ (Connector Framework - CF) Exchange public folders (CF) External systems(CF) Documentum™ content (CF)

38. Fast Search out-of-box Connectors Enterprise crawler Good for large scale crawling across multiple nodes Supports dynamic data including JavaScript JDBC™ connector Simple configuration using SQL commands Supports push based crawling Supports change detection through checksums

39. SharePoint Connector Accesses data through SharePoint web service (using Windows credentials) Supports full crawl through enumeration of content and incremental crawl through change log Built-in support for security trimming (Windows and pluggable)

40. Connector Framework Easy to build Write managed code instead of PH interface Built connectors shared across SharePoint Search and FAST search ‘No-code’ solution for Databases and Web Services Easy to manage Use existing user interfaces for crawl configuration Faster iteration on connectivity using SharePoint Designer (SPD) Shared advantages across SharePoint Can use other BCS features such as external lists

41. Search page points to Profile Page or custom page to show search result Search results page – results based on metadata Connector Framework Architecture Search user queries Search Engine/Index BCSProfile Pages or External Lists Search BCS Runtime Execution call routing through BCS Connectors using BCS model file Fetch data from repository WCF Databases .NET BCSConnector Custom BCSConnector Protocol Handlers

42. Enabling Connectivity Every indexing connector needs A BDC model file to express connection information and the structure of the backend A BDC Connector for code to execute when accessing the backend Out-of-box BDC Connectors Database WCF (Web Services) .NET (custom code)

43. Federation Combining Search Results

44. Eliminate Compromise Easily Connect to Corporate Assets that live Outside of SharePoint Files OpenSearch Federation Web Databases SharePoint Easy to Connect Easy to Build Exchange Server Enterprise Content Mgmt Systems Line of Business Apps Custom Apps Asset Mgmt systems Documentum Indexing Connectors Lotus Notes …

45. “Out of the Box” Federation People Search and Related Searches Bing and other OpenSearch providers Windows 7 and IE8 searching SharePoint 2010 Rich RSS feeds and federation generator

46. How OpenSearch federation works HTTP request Search Client Search Provider RSS/Atom Results HTTP request with query in the URL: http://www.site.com/srchrss.aspx?q={searchTerms} RSS / Atom results: RSS results with <title>, <link>, <description> Best sources also include: <pubdate>, <author>, <category>, <media:thumbnail> Optionally include custom metadata: <recordid>, <projectname>, <contactnumber>

47. Deploying a federation connectorUse or create a searchable feed Search Client OpenSearchPage Search Provider Use an existing searchable feed or create your own. Anything searchable– takes a query as a URL parameter. Must return results in RSS or Atom formatted XML. Creating a searchable feed is simple. Just code a basic ASPX page.

48. Make Search Pervasive!Deploy OpenSearch (OSDX files) Everywhere SharePoint 2010: Import the file. Add it to a Search Center. Windows 7: Pull: Add a link to the OSDX file on the search center or team site. Push: Deploy the OSDX file via Group Policy. IE8 Link to Search Provider Discoverable Search Provider

49. Extend and Create Search Web Parts The OOB Web Parts are no longer sealed!

50. Microsoft Enterprise Search - Productivity Products for Every Search Need

51. FAST Search for SharePoint and ESP A new choice; a different world Bringing high-end search to the mainstream market

52. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

54. End User perspective Go Beyond the Search Box

55. IT Pro perspective Eliminate Compromise

56. Do More with Search Developer Perspective

57. Search Technology Concepts Concepts Search Center - UI for users to issue queries and interact with results Federated Source Query Object Model Query Servers- Accept query requests from users and return results Index Files Query Servers Query Federation - Return results from non-SharePoint Indexes Indexing - Extract information from items to enable efficient matching Indexer Index Partition - Subset of the overall index Crawling - Traverse URL space to record items in search catalog Crawler Indexing Connectors - Know how to process different content sources Content Sources - Host the content we want to return in main results Content Content Content