Session from SPSNE outlining what intelligent search is, what's OOB, and how to go way beyond OOB

1. Jeff Fried
CTO, BA Insight
Use Office 365 and Azure Cognitive Services
for Intelligent Search
October 2017

2. Longtime
Search Nerd
• CTO, BA Insight
• Senior PM, Microsoft
• VP, FAST
• SVP, LingoMotors
Jeff Fried
Passionate About
• Search
• Search-driven applications
• Information Strategy
• Text Analytics
• Machine Learning
Email: jeff.fried@bainsight.com
Twitter: @jefffried
Blog: www.DoMoreWithSearch.com

4. Outline

5. Wait, isn’t search
ALREADY ‘intelligent’?

6. Matching IntentContent

7. Content Intent
Lookup
Learn
ExploreMessy
Human
Language

8. A new wave of Intelligent Search
Engines?
Machine Learning (ML), Natural Language Processing (NLP),
Semantics, Cognitive, Artificial Intelligence (AI), Self-Learning, …
http://www.cmswire.com/information-management/intelligent-cognitive-ai-based-search-separating-hype-from-reality/

9. Forrester: “Keyword search of the past has evolved into Cognitive Search”
Forrester, June 2017

10. Evolution of enterprise search technology
TODAYYESTERDAY TOMORROW
Relevancy based on general criteria that
compare how well a single item matches a
query, compared to the rest of the index.
Matching Content & Popularity
By leveraging your personal activity, we
can signal highly relevant content that
would otherwise drown in the noise
Personalized & behavior based Drive actionable insights
Take action on data and empower
decision making by understanding
intention across many facets
Scope & scale of Microsoft Graph
Use machine learning to create
insights from your data
Construct pairs for questions & answers
Query match quality in content or page
Popularity of document or page
Content metadata such as created date
Bing entity extraction
Understanding organizational priorities
as well as your extended network
Using AI for semantic understanding
+ +

11. 11
Semantics! Machine Learning! AI!

12. Latest Enhancements to SPO search

14. Making O365 search intelligent using Azure Cognitive Services
more
ˇ
Start with understanding how it works

15. Matching IntentContent
Context
Content
Enrichment
UX
Connections
Query
Augmentation

16. Insight Data Sets
Trending
Documents
People I
Work With
Used
Documents
Email
Files
Contacts
Groups
Meetings
Org
Tasks
File Enrichments
Item
Metadata
# Activity
History
myGraph
Sites
I Use
Suggested
Sites
Usage
Statistics

17. How does the search box work?
Used
Documents
Trending
Documents
Sites
I Use
Suggested
Sites
Beautiful
Bergen
Contoso
Norway
Kathrine
Hammervold

18. Metadata Drives Great User Experiences
Documents from many sources
All client or matter-relevant documents are integrated. Rich MetaData
Content annotated automatically – concepts,
categories, citations, matters, clients, etc
Navigation Controls
Explore, Discover, Drill-down

19. Where is the Metadata in Delve?

20. So….Idon’tneed
metadata,right?
Delve doesn’t have refinement
Chart courtesy of xkcd

21. Outline

22. The Cognitive Arms Race

23. Agent Bots Cognitive Machine Learning Augmented Reality
Microsoft Cortana Bot Framework Cognitive Services
- Vision
- Speech
- Language
- Knowledge
- Search
- Labs
Azure ML
Cognitive Toolkit
Hololens
Google Google Assistant API.AI Cloud Vision API
Cloud Video Intelligence
Cloud Speech API
Cloud Natural Language API
Google Knowledge Graph
Google Custom Search
ML Advanced Solutions Lab
Cloud ML Engine
TensorFlow
Glass at Work
Daydream/Tango
IBM Watson Virtual Agent Watson Conversation Watson APIs
- Vision
- Speech
- Language
- Data Insights
Watson ML Service
Apache SystemML
Amazon Alexa Amazon Lex Rekognition
Polly
Amazon ML
Apple Siri SiriKit
Speech
Core ML ARKit
Facebook Facebook M ParlAI FastText
CommAI
Torch Oculus Rift
The Cognitive Arms Race

24. Matching IntentContent
Context
Content
Enrichment
UXConnections
Query
Augmentation
Making SharePoint search intelligent with Cognitive Services
more
ˇ

26. BA Insight Product Portfolio
Smart Analytics
AutoClassifier
Content
from 60+
Enterprise
Systems
Cognitive Services (AI, Machine Learning)
Search
Index(es)
Search for
Dynamics
Connectors
Expertise
Locator
Smart
Previews
SmartHub

27. Content
Content
Processing
Search
Index
/ Graph
Search UI
Enhancing Content with Cognitive Services
Query
Processing
Content
Enrichment

28. AutoClassifier
rules-driven core, combined with learning-based modules
Enhanced Content
Enriched with
Metadata and
Content Types
Search Visualization Workflow

29. Adding Text Analytics to Content
User
Uploads
Content Event
Fires
Content
Text
Analysis
Sentiment - Is text positive or
negative?
Key Phrases - What are
people discussing in a single
article?
Topics - What are people
discussing across many
articles?
Language - What language is
text written in?
Add
metadata
to item
Shows in
O365
search

30. Adding Translation to Content
User
Uploads
Content Event
Fires
Content
Text
Analysis
Translation – Copy existing
content and translate to
specified language
save in
SharePoint
Shows in
O365
search
Create new file/content for
specific language

31. Image Search
User
Uploads
Content Event
Fires
Computer
Vision
Image Processing – recognize
objects and text inside
images
Shows in
O365
search
Add
metadata
to item
Extract
image(s)
from
documents

32. Image Classifier and Search
project libraries
and
issue detection
Search Visualization Workflow

34. Feed content from over
60 enterprise systems
Search
Index(es)
Search UI
Connectors
Smart
Analytics
Extensible
content processing
Pluggable query and
results pipelines
Auto
Classifier
Cognitive
Services
SmartHub
“Architectural” View

36. 37
Content is spread across lots of systems
• Aderant
• Amazon S3
• Alfresco
• Box
• Confluence
• CuadraSTAR
• Elite / 3E
• EMC Documentum
• EMC eRoom
• Google Drive
• HP Consolidated Archive
• (EAS, aka Zantaz)
• HPE Records Manager/HP TRIM
• IBM Connections
• IBM Content Manager
• IBM DB2
• IBM FileNet P8
• IBM Lotus Notes
• IBM WebSphere
• iManage Work
• Jive
• LegalKEY
• LexisNexis Interaction
• Lotus Notes Databases
• Microsoft Dynamics CRM
• Microsoft Exchange
• Microsoft Exchange Public Folders
• Microsoft SQL Server
• MySQL
• NetDocuments
• Neudesic The Firm Directory
• Objective
• OpenText LiveLink/RM
• OpenText eDOCS DM
• Oracle Database
• Oracle WebCenter
• Oracle WebCenter Content (UCM/Stellent)
• PLC/Practical Law
• ProLaw
• Salesforce.com
• SAP ERP
• ServiceNow
• SharePoint Online
• SharePoint 2016
• SharePoint 2013
• SharePoint 2010
• SharePoint 2007
• Sitecore
• Any SQL-based CRM system
• Veeva Vault
• Veritas Enterprise Vault
(Symantec eVault)
• West km
• Xerox DocuShare
• Yammer
List of standard connectors, plus a proven architecture and
process for creating new connectors to complex systems

38. Feed content from over
60 enterprise systems
Search
Index(es)
Search UI
Connectors
Smart
Analytics
Extensible
content processing
Pluggable query and
results pipelines
Auto
Classifier
Cognitive
Services
SmartHub
Bots
AR (hololens) apps
Bots and AR

40. Better Search
- data enrichment
- query augmentation
- relevance
- recommendations
New Types of Search
and Discovery
- multi-media search
- multi-lingual search
- spoken language search
- search bots
- search in AR
AI for Intelligent Search & Discovery

41. Content
Content
Processing
Search
Index
Search UI
ML applied to search
Query
Processing
Learning about content structure Learning about user intent
Analytics
Behavioral data to drive learning
Learning to rank

42. BA Insight Product Portfolio
Smart Analytics
AutoClassifier
Content
from 60+
Enterprise
Systems
Cognitive Services (AI, Machine Learning)
Search
Index(es)
Search for
Dynamics
Connectors
Expertise
Locator
Smart
Previews
SmartHub

44. www.BAinsight.com
Jeff.Fried@BAinsight.com
@jefffried