Access Services in SharePoint 2010 - All You Need to Know
Building Search Driven Applications in SharePoint 2010 - SharePoint Fest 2012
1. Building Search Driven Applications
in SharePoint 2010
Nik Patel
Lead Solutions Architect, Slalom Consulting, Chicago Practice
2. Session Goals
Audience – SharePoint Architects, SharePoint Developers, and Power Users
Main Goal - Understanding Search from Application Development Perspective
What this Session will cover
Out of Box Search Configurations
On-Premises Enterprise Search API
What this Session won’t cover
FAST Search API
Search in SharePoint Online (First half partially applies to Office 365)
2
3. About Me
Nik Patel
• Lead Solutions Architect, Slalom Consulting
• Working with SharePoint 2010 since Summer 2009 TAP
• Master in Computer Science from IIT, Chicago
Contact Info
• Email – patenik2@yahoo.com
• Blog – Nik Patel’s SharePoint World -
http://nikspatel.wordpress.com/
• Twitter - @nikxpatel
• LinkedIn - linkedin.com/in/nikspatel
• Slideshare - slideshare.net/patenik2
3
4. Session Agenda
Why Search is Important in SharePoint Information Architecture?
Why Search is Important as SharePoint Data Access Method?
Overview of SharePoint Enterprise Search Architecture
Out of Box Customization Options for SharePoint 2010 Search Applications
Demo – Custom Search Solutions using OOB Web Parts
Programming Options for SharePoint 2010 Search Applications
Demo – Custom Search Solutions using Keyword Search API
Questions and Answers
4
5. Why Search and Metadata Driven Information Architecture necessary?
Information is everywhere, Needs foundational architecture to find it efficiently
Information Islands
Modern intranet has Information and Applications Islands and Needs to find information efficiently
Find and Target Relevant Information
User can receive information if they are tagged property. Search can leverage metadata to target relevant
information
Aggregate information from Distributed SharePoint Information Architecture
Rollup information across Multiple Sites, Site Collections, Web Applications, and Farms
Next time when you design systems, think about Search + Metadata Driven
Information Architecture
5
6. Why Search is Important as SharePoint Data Access Method?
Overview and Limitation of Traditional SharePoint 2010 Data Access APIs
• Server Side APIs
• SPList, SPQuery, and SPDataQuery
• LINQ to SharePoint
• Client Side and Remote APIs Client OM REST APIs SOAP APIs
• CSOM and REST APIs
• SOAP APIs and WCF Services
• Targeted at List or Library Data Platform
• Scoped at the Site or Site Collection Farm Site List Data External Lists
• 5000 List Items threshold
• CSOM has slow performance over
the network
• Data Aggregation and Rollup would
require complex custom
programming
6
7. Why Search is Important as SharePoint Data Access Method?
Enterprise Search delivers Data Rollups and Data Aggregations efficiently
• Use OOB Web parts, Search APIs, or Search ASMX Services to query data against Search Index
• Major Benefits
• Perfect for Data Rollups and Data Aggregations across Cross-farm, Cross-Web Application, or Cross-Site
Collection
• Can query across large data corpuses (because it query against index)
• Queries are fast even with large data corpuses (because it query against index)
• Most scalable data access solution out there, All other techniques are mostly SQL-based (because it query
against index)
• Major Limitations
• Content needs to be in the indexed (dependency), Avoid this option if real time data access are requirements
• Requires advanced search service administration and configuration (dependency)
• Improve search by tagging with metadata (to be effective)
7
8. Typical Search Application Scenarios and Business Cases
Typical Business Cases
De-centralized content, data, and documents
Large amount of data, document libraries, sites, or site
collections
IA Governance – Metadata, Content Types, and Search
Typical Application Scenarios
Cross Site Collection Portal Navigation
My Team Sites Dashboard
Corporate Knowledge Center - Rolling up documents
Customer Documents Dashboard - Cross Web
Application or Cross-Farm Documents Sharing for
Extranets
8
10. Enterprise Search Terminology for Developers – Major Puzzle Pieces
Understanding out of box crawling and indexing pipeline would benefit
Content Sources
Crawling Schedules
Crawl Rules
Content Scopes
Crawled Properties
Managed Properties
Search Query Object Model
Enterprise Search Center
Search by default is Security Trimmed
10
11. Out of Box No-Code SharePoint Search Customization
Audience – Power Users & SharePoint Architects
11
12. Why would you Customize Search?
Out of box Search configuration doesn’t cover all business cases
Typical complains with OOB Search Center
Too many results – They are not relevant
Missing information on search page
Improving how Users View results
OOB interface may not provide ideal UI
Tailoring search center may greatly increase
productivity or usability
Tip - Try using Out of box Options
before exploring Custom Coding
Options
12
13. Out of Box Search Capabilities – Search Web Parts
Search Web Parts can be used by Power Users to build Search Applications
Major components in configuring No- Advanced Search Box Allows users to create detailed searches against
Managed Properties
Code browser based Search Applications
Federated Results Displays results from a federated search location
Presents facets that can be used to refine a people
People Refinement Panel
search
People Search Box Allows users to search for people using a keyword
People Search Core Displays the primary result set from a people
Results search
Displays a summary of search results and enables
Refinement Panel
users to filter results.
Displays the related searches for a particular user
Related Queries
query.
Displays links for RSS, alerts, and Windows
Search Action Links
Explorer
Search Best Bets Displays best-bets results
Search Box Allows users to enter keyword query searches
Search Core Results Displays the search results for a user's query
Search Paging Allows a user to page through search results
Presents statistics for search results like total
Search Statistics
search results
Search Summary Displays the summary for a user's query.
Top Federated Results Displays top results from a federated location
14. Out of Box Search Capabilities – Core Results Web Part
Displays search results based on user query or predefined queries
Use in conjunction with Refiner Panel,
Search Box, Search Paging, and Search
Statistics web parts
Major features
Displays search results
Specify a scope or content source to limit
results
Customize what fields returned by Search
Results customizable using XSL
Can hard code query on behalf of the user
Use the KeywordQuery API syntax
Fixed Keyword Query
Append Text to Query
More Info - http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/04/05/a-quick-guide-to-coreresultswebpart-configuration-changes-in-sharepoint-2010.aspx
15. Out of Box Search Capabilities – Refiner Panel Web Part
Refines search results in Core Results web part
Use in conjunction with Core Results,
Search Box, Search Paging, and Search
Statistics web parts
Major Features
Refine results based upon categories, metadata,
managed metadata or tags
Customizable using XSL
Filter categories configured using XML
Accuracy Index – Refiners results based on first 50
results by default,
Beware of Use Default Configuration – Uncheck for
custom filter category definitions
More Info: http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/03/15/get-to-know-the-refinement-web-part-in-sharepoint-2010-enterprise-search.aspx
16. Improve Search with Queries - Keyword Query Syntax
Refines and filter query results by Keyword Query Syntax
Improved Keyword Syntax Operators
in SharePoint 2010
Property Operators: :, =, <, >, <=, >=, <>
Boolean Operators: AND, NOT, and OR
Wildcard Operator: *, Only supports prefix
matching e.g. account* returns data with account,
accounts, accountant etc.
Proximity Operators: x NEAR y (up to 8 terms
away)
Synonym Operators: WORDS(TV, Television),
As opposed to “TV OR Television”, WORDS
ranks them equivalently
Inclusion/Exclusion Operators: +/-
Parenthesis Support for Complex Grouping of
queries
17. Improve Search with Queries - Keyword Query Syntax (Continued)
Refines and filter query results by Keyword Query Syntax
Streamline with Managed Properties
Search by either built-in or custom managed
properties
Managed properties must be configured in
Search Center against crawled properties
Sample OOB Managed Properties
ContentSource:"Local SharePoint Sites“
Scope:”Products”
Site:”http://sp2010vm/products”
ContentType:”Policy Type”
isDocument:0 (0 or 1)
ContentClass:STS_ListItem_Tasks (e.g. STS_Site,
STS_Web, STS_List, STS_ListItem, STS_List_{Type},
STS_ListItem_{Type})
FileType:“docx"
20. Why would you Extend Search or Build Custom Search Solutions?
Building Custom Search Applications using Search APIs
Typical Scenarios
Out-of-the-box search works great, but users want a rich interface
Out-of-the-box search doesn’t meet all the business requirements and requires advanced custom logic
Accessing SharePoint Search Index remotely from the ASP.NET, Silverlight, or Mobile Applications
Typical Options
Extend OOB Search Results Web Part
Programming against Server Side Object Model
Keyword Query API
FullTextSQL Query API
Programming against Search Web Services (Not Covered Today)
SharePoint Search SOAP API - http://sp2010vm/_vti_bin/spsearch.asmx
Enterprise Search SOAP API: http://sp2010vm/_vti_bin/search.asmx
NOTE: Search is not supported in SharePoint 2010 CSOM API
More info - Enterprise Search Query Architecture - http://msdn.microsoft.com/en-us/library/ee558338.aspx
20
21. Extending the OOB Core Results Web Part
Simplest way to extend out of box functionality
Why? How?
Dynamically set search query on behalf of Create new web part, extend CoreResultsWebPart
the user
Specify Fixed query and Additional query
Set key properties on derived class
Location (Content Source) & Scope (Specify Scope)
User interactions (e.g. toggling custom views based
on user friendly search queries) FixedQuery & AppendedQuery
Reading query from non-traditional source (e.g. SelectColumns – All rendered columns must be in this XML string
Query string, custom formula)
Xsl & XslLink
Dynamically update look and feel Modify the Query Pipeline
Search by default sort results by relevance, Instead
apply custom sorting Override GetXPathNavigator Method
Apply different custom XSLTs for different layouts QueryManager class allows us to intercept query pipeline before
based on user queries and after query is issued
More Info - How to: Use the QueryManager class to query SharePoint 2010 Enterprise Search -
http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/08/15/how-to-use-the-querymanager-class-to-query-sharepoint-2010-enterprise-search.aspx
22. Server Side Query Object Model - Search Query APIs
KeywordQuery API, FullTextSQLQuery API, and Fast Query Language (FQL)
KeywordQuery API FullTextSQLQuery API
Preferred Approach for Search Queries in SP2010 Executes Search SQL Syntax queries
Executes Keyword Query Syntax queries Executes queries against Search Scopes
Query executes against Search Index Doesn’t require managed properties
Requires content must be crawled, crawled Puts load on SQL rather than Query servers
properties must be in the index, and managed
properties must be mapped to crawled Use if you prefer the SQL syntax and execute full
properties. text query programmatically
FAST doesn’t support SQL Syntax, use Keyword Unlike Keyword Query API, SQL Syntax support
Query Syntax to seamlessly support code in CONTAINS parameter
both Enterprise Search and FAST
22
23. Step by Step – Using Search Query APIs
Initial steps are same for KeywordQuery API and FullTextSQLQuery API
Step 1 - Create SharePoint 2010 Visual Studio Solution based on SharePoint 2010 Project Template
Step 2 - Add SharePoint Project Item for Search Application UI & Logic
Custom Web Part or Visual Web Part
Custom Application Page or Site Page
Step 3 - Reference two Search DLLs from the ISAPI directory
~SharePointRootISAPIMicrosoft.Office.Server.dll
~SharePointRootISAPIMicrosoft.Office.Server.Search.dll
Typical SharePoint Root is - C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions14
Step 4 - Import Search Namespaces
using Microsoft.Office.Server;
using Microsoft.Office.Server.Search;
using Microsoft.Office.Server.Search.Query; [KeywordQuery Class defined in this namespace]
using Microsoft.Office.Server.Search.Administration;
24. Step by Step – Using KeywordQuery API
• Step 1 - Instantiate the KeywordQuery object
• You have two options, pass current site collection (Classic MOSS 2007 way), or pass the Search Query and
Site Settings Service proxy (New in SP2010) and instantiate KeywordQuery object
1. SearchQueryAndSiteSettingsServiceProxy settingsProxy = SPFarm.Local.ServiceProxies.GetValue
<SearchQueryAndSiteSettingsServiceProxy>();
SearchServiceApplicationProxy searchProxy =
settingsProxy.ApplicationProxies.GetValue <SearchServiceApplicationProxy>("Search Service
Application");
KeywordQuery q = new KeywordQuery(searchProxy);
2. SearchServiceApplicationProxy proxy =
(SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(SPCont
ext.Current.Site));
KeywordQuery q= new KeywordQuery(proxy);
25. Step by Step – Using KeywordQuery API (Continued…)
• Step 2 - Set KeywordQuery Object Properties
• QueryText – This is keyword query text (e.g. LastName:”Patel”)
• ResultTypes – Search result types (default is Relevant Results)
• HiddenContraints – Filter by search scope
• RowLimit – By default, it returns only 50 records, Se this property to increase number
• RowsPerPage - Paging
• IgnoreAllNoiseQuery
• TrimDuplicates – Trims duplicate results
• EnableNicknames
• EnablePhonetic
• EnableStemming
• EnableSpellCheck
• EnableFQL
26. Step by Step – Using KeywordQuery API (Continued…)
• Step 3 – Use SelectProperties string collection to return custom managed properties
• Out of the box, KeywordQuery API will return only default search properties such as Title, Author etc.
• To return managed properties, use SelectProperties string collection
• Step 4 - Run KeywordQuery Object Execute() method and return ResultTableCollection which can be
loaded in ADO.NET DataTable Object
• Step 5 - Bind ADO.NET DataTable Object data into ASP.NET or SharePoint Controls for UI
More Info from Corey Roth’s blog
How to: Use the MOSS Enterprise Search KeywordQuery class - http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/02/19/how-to-
use-the-moss-enterprise-search-keywordquery-class.aspx
How to: Use the SharePoint 2010 Enterprise Search KeywordQuery Class -
http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2010/08/12/how-to-use-the-sharepoint-2010-enterprise-search-keywordquery-
class.aspx
27. Step by Step – Using KeywordQuery API (Complete Example)
29. Step by Step – Using FullTextSQLQuery API
• Step 1 – Just like KeywordQuery API, Instantiate the FullTextSqlQuery object
• Just like KeywordQuery API, You have two options, pass current site collection (Classic MOSS 2007 way), or
pass the Search Query and Site Settings Service proxy (New in SP2010) and instantiate FullTextSqlQuery object
1. SearchQueryAndSiteSettingsServiceProxy settingsProxy = SPFarm.Local.ServiceProxies.GetValue
<SearchQueryAndSiteSettingsServiceProxy>();
SearchServiceApplicationProxy searchProxy =
settingsProxy.ApplicationProxies.GetValue <SearchServiceApplicationProxy>("Search Service
Application");
FullTextSqlQuery q = new FullTextSqlQuery(searchProxy);
2. SearchServiceApplicationProxy proxy =
(SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(SPCont
ext.Current.Site));
FullTextSqlQuery q= new FullTextSqlQuery(proxy);
30. Step by Step – Using FullTextSQLQuery API (Continued…)
• Step 2 - Set FullTextSQLQuery Object Properties
• QueryText – This is SQL Query text, You can specify scopes in the query
• ResultTypes – Search result types
• RowLimit – By detault, it returns only 50 records, Se this property to increase number
• RowsPerPage - Paging
• IgnoreAllNoiseQuery
• TrimDuplicates
• EnableNicknames
• EnablePhonetic
• EnableStemming
• Step 3 - Run FullTextSQLQuery Object Execute() method and return ResultTableCollection which can
be loaded in ADO.NET DataTable Object
• Step 4 - Bind ADO.NET DataTable Object data into ASP.NET or SharePoint Controls for UI
• Step 5 – Unlike KeywordQuery Object, you must dispose FullTextSQLQuery object
How to: Use the MOSS Enterprise Search FullTextSqlQuery Class - http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2008/03/06/how-to-
use-the-moss-enterprise-search-fulltextsqlquery-class.aspx
31. Step by Step – Using FullTextSQLQuery API (Complete Example)
32. • Thanks for attending the session!!
• Follow SharePoint Fest Chicago
• Twitter - @SharePointFest - https://twitter.com/sharepointfest
• Website - http://www.sharepointfest.com/Chicago/
32