SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Downloaden Sie, um offline zu lesen
The Future of Search & AI
Trey Grainger
SVP of Engineering, Lucidworks
@treygrainger
#Activate18 #ActivateSearch
612ATTENDEES
39COUNTRIES
70SESSIONS
So what are the current and 

upcoming trends in Search & AI?
Big Data?
We need big answers.
Because we don’t need “big data.”
Solr leads the way
on Learning to Rank
And that mass adoption of LTR looks like this..
But what about all the other aspects
of Search beyond AI?
relevance, scale, bm25, user experience, learning to rank,
autoscaling, sys admin, server provisioning, lemmatization/stemming,
cloud, docker, aws, natural language processing, etl, entity extraction,
tf-idf, phrase detection, semantic search, query pipelines, connectors,
web crawling, signals, real-time recommendations, personalization,
question answering, request handlers, search components,
autocomplete, spell checking, synonyms, stop words, query-log
mining, probabilistic query parsing, boolean queries, geo-spatial
queries, image search, voice search, classification models, bi-
directional cross-datacenter replication, semantic knowledge graphs,
ontology management, merchandizing, entity type detection, query
classification, word embeddings, query parsers, highlighting, analyzers,
tokenization, streaming expressions, faceting, concept clustering,
more like this, insight engine, apache lucene/solr, lucidworks fusion,
some kind of crazy magic, that white box with the magnifying glass...
What Is Search?
There are a LOT of skillsets required to
deliver a world-class Search experience
Scaling search can be really hard
Ease of operations, scaling, and
performance are a top priority
We're drastically improving ease of operations
Time factors
•
And making the cluster aware of its own needs
Size factors Budget factors
• Scaling Policies
• Server Metrics
• Event Triggers
• Disk Space Free
• Prioritizing Traffic
• CPU Too High
• Traffic Increasing
• Need more Nodes
• Shards out of balance
• Back Pressure
• Traffic Increasing
• Server Lost
or.
How I indexed *.
... and lived to tell the tale ....
* simulated.
1.000.000.000.000
documents.
Final Thoughts
There is an ongoing shift away from "big data"
and toward actionable intelligence
Assistive Search is the new paradigm
•Requires personalization
•Voice
•Image Search
•Conversational
•Contextual (location, last search)
•Beyond links and information to 

Answers and Action
... and Search is now the de-facto user interface for 

digital interaction with your customers
The future of business success hinges on the
outcomes of billions of digital moments
FIND 

the correct
product
CONNECT 

to the right
person KNOW
exactly what
to do next
DRIVE
relevant
conversations
RESOLVE
a problem
The Future of Search & AI
Starts Now.
Thank you!
@treygrainger
#Activate18 #ActivateSearch

Weitere ähnliche Inhalte

Was ist angesagt?

Reflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data systemReflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data system
Trey Grainger
 
Balancing the Dimensions of User Intent
Balancing the Dimensions of User IntentBalancing the Dimensions of User Intent
Balancing the Dimensions of User Intent
Trey Grainger
 
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Trey Grainger
 
The Apache Solr Smart Data Ecosystem
The Apache Solr Smart Data EcosystemThe Apache Solr Smart Data Ecosystem
The Apache Solr Smart Data Ecosystem
Trey Grainger
 
The Semantic Knowledge Graph
The Semantic Knowledge GraphThe Semantic Knowledge Graph
The Semantic Knowledge Graph
Trey Grainger
 
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey GraingerHaystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
OpenSource Connections
 

Was ist angesagt? (20)

Reflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data systemReflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data system
 
AI, Search, and the Disruption of Knowledge Management
AI, Search, and the Disruption of Knowledge ManagementAI, Search, and the Disruption of Knowledge Management
AI, Search, and the Disruption of Knowledge Management
 
Balancing the Dimensions of User Intent
Balancing the Dimensions of User IntentBalancing the Dimensions of User Intent
Balancing the Dimensions of User Intent
 
Measuring Relevance in the Negative Space
Measuring Relevance in the Negative SpaceMeasuring Relevance in the Negative Space
Measuring Relevance in the Negative Space
 
The Intent Algorithms of Search & Recommendation Engines
The Intent Algorithms of Search & Recommendation EnginesThe Intent Algorithms of Search & Recommendation Engines
The Intent Algorithms of Search & Recommendation Engines
 
South Big Data Hub: Text Data Analysis Panel
South Big Data Hub: Text Data Analysis PanelSouth Big Data Hub: Text Data Analysis Panel
South Big Data Hub: Text Data Analysis Panel
 
Crowdsourced query augmentation through the semantic discovery of domain spec...
Crowdsourced query augmentation through the semantic discovery of domain spec...Crowdsourced query augmentation through the semantic discovery of domain spec...
Crowdsourced query augmentation through the semantic discovery of domain spec...
 
Thought Vectors and Knowledge Graphs in AI-powered Search
Thought Vectors and Knowledge Graphs in AI-powered SearchThought Vectors and Knowledge Graphs in AI-powered Search
Thought Vectors and Knowledge Graphs in AI-powered Search
 
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
Searching on Intent: Knowledge Graphs, Personalization, and Contextual Disamb...
 
The Apache Solr Semantic Knowledge Graph
The Apache Solr Semantic Knowledge GraphThe Apache Solr Semantic Knowledge Graph
The Apache Solr Semantic Knowledge Graph
 
The Apache Solr Smart Data Ecosystem
The Apache Solr Smart Data EcosystemThe Apache Solr Smart Data Ecosystem
The Apache Solr Smart Data Ecosystem
 
Self-learned Relevancy with Apache Solr
Self-learned Relevancy with Apache SolrSelf-learned Relevancy with Apache Solr
Self-learned Relevancy with Apache Solr
 
The Semantic Knowledge Graph
The Semantic Knowledge GraphThe Semantic Knowledge Graph
The Semantic Knowledge Graph
 
Reflected intelligence evolving self-learning data systems
Reflected intelligence  evolving self-learning data systemsReflected intelligence  evolving self-learning data systems
Reflected intelligence evolving self-learning data systems
 
Leveraging Lucene/Solr as a Knowledge Graph and Intent Engine
Leveraging Lucene/Solr as a Knowledge Graph and Intent EngineLeveraging Lucene/Solr as a Knowledge Graph and Intent Engine
Leveraging Lucene/Solr as a Knowledge Graph and Intent Engine
 
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey GraingerHaystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
Haystack 2019 - Natural Language Search with Knowledge Graphs - Trey Grainger
 
Enhancing relevancy through personalization & semantic search
Enhancing relevancy through personalization & semantic searchEnhancing relevancy through personalization & semantic search
Enhancing relevancy through personalization & semantic search
 
Building Search & Recommendation Engines
Building Search & Recommendation EnginesBuilding Search & Recommendation Engines
Building Search & Recommendation Engines
 
Intent Algorithms: The Data Science of Smart Information Retrieval Systems
Intent Algorithms: The Data Science of Smart Information Retrieval SystemsIntent Algorithms: The Data Science of Smart Information Retrieval Systems
Intent Algorithms: The Data Science of Smart Information Retrieval Systems
 
Extending Solr: Building a Cloud-like Knowledge Discovery Platform
Extending Solr: Building a Cloud-like Knowledge Discovery PlatformExtending Solr: Building a Cloud-like Knowledge Discovery Platform
Extending Solr: Building a Cloud-like Knowledge Discovery Platform
 

Ähnlich wie The Future of Search and AI

Activate 2018 Closing Remarks: The Future of Search & AI - Trey Grainger
Activate 2018 Closing Remarks: The Future of Search & AI - Trey GraingerActivate 2018 Closing Remarks: The Future of Search & AI - Trey Grainger
Activate 2018 Closing Remarks: The Future of Search & AI - Trey Grainger
Lucidworks
 
SPLive Orlando - Beyond the Search Center - Application or Solution?
SPLive Orlando - Beyond the Search Center - Application or Solution?SPLive Orlando - Beyond the Search Center - Application or Solution?
SPLive Orlando - Beyond the Search Center - Application or Solution?
Agnes Molnar
 
3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию
antishmanti
 
Mastering Customer Data on Apache Spark
Mastering Customer Data on Apache SparkMastering Customer Data on Apache Spark
Mastering Customer Data on Apache Spark
Caserta
 
Spca2014 building a digital workplace on share point sam marshall 1.0d
Spca2014 building a digital workplace on share point   sam marshall 1.0dSpca2014 building a digital workplace on share point   sam marshall 1.0d
Spca2014 building a digital workplace on share point sam marshall 1.0d
NCCOMMS
 

Ähnlich wie The Future of Search and AI (20)

Activate 2018 Closing Remarks: The Future of Search & AI - Trey Grainger
Activate 2018 Closing Remarks: The Future of Search & AI - Trey GraingerActivate 2018 Closing Remarks: The Future of Search & AI - Trey Grainger
Activate 2018 Closing Remarks: The Future of Search & AI - Trey Grainger
 
How to crack Big Data and Data Science roles
How to crack Big Data and Data Science rolesHow to crack Big Data and Data Science roles
How to crack Big Data and Data Science roles
 
Frank Bien Opening Keynote - Join 2016
Frank Bien Opening Keynote - Join 2016Frank Bien Opening Keynote - Join 2016
Frank Bien Opening Keynote - Join 2016
 
Frank Bien Opening Keynote - Join 2016
Frank Bien Opening Keynote - Join 2016Frank Bien Opening Keynote - Join 2016
Frank Bien Opening Keynote - Join 2016
 
Graph Thinking: Why it Matters
Graph Thinking: Why it MattersGraph Thinking: Why it Matters
Graph Thinking: Why it Matters
 
How AI-Powered Search Drives Employee Experience
How AI-Powered Search Drives Employee ExperienceHow AI-Powered Search Drives Employee Experience
How AI-Powered Search Drives Employee Experience
 
Crawlable Spatial Data - #Geo4Web research topic #3
Crawlable Spatial Data - #Geo4Web research topic #3Crawlable Spatial Data - #Geo4Web research topic #3
Crawlable Spatial Data - #Geo4Web research topic #3
 
BIBoss: The Data Science Behind Personalisation & AI
BIBoss: The Data Science Behind Personalisation & AIBIBoss: The Data Science Behind Personalisation & AI
BIBoss: The Data Science Behind Personalisation & AI
 
SPLive Orlando - Beyond the Search Center - Application or Solution?
SPLive Orlando - Beyond the Search Center - Application or Solution?SPLive Orlando - Beyond the Search Center - Application or Solution?
SPLive Orlando - Beyond the Search Center - Application or Solution?
 
Artificial Intelligence disruption: How technologies are predicted to change ...
Artificial Intelligence disruption: How technologies are predicted to change ...Artificial Intelligence disruption: How technologies are predicted to change ...
Artificial Intelligence disruption: How technologies are predicted to change ...
 
AI in Talent Acquisition - Talent Connect 2017
AI in Talent Acquisition - Talent Connect 2017AI in Talent Acquisition - Talent Connect 2017
AI in Talent Acquisition - Talent Connect 2017
 
Big Data Evolution
Big Data EvolutionBig Data Evolution
Big Data Evolution
 
3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию3 джозеп курто превращаем вашу организацию в big data компанию
3 джозеп курто превращаем вашу организацию в big data компанию
 
Mastering Customer Data on Apache Spark
Mastering Customer Data on Apache SparkMastering Customer Data on Apache Spark
Mastering Customer Data on Apache Spark
 
Webinar: Enterprise Search in 2025
Webinar: Enterprise Search in 2025Webinar: Enterprise Search in 2025
Webinar: Enterprise Search in 2025
 
SDSC18 and DSATL Meetup March 2018
SDSC18 and DSATL Meetup March 2018 SDSC18 and DSATL Meetup March 2018
SDSC18 and DSATL Meetup March 2018
 
Spca2014 building a digital workplace on share point sam marshall 1.0d
Spca2014 building a digital workplace on share point   sam marshall 1.0dSpca2014 building a digital workplace on share point   sam marshall 1.0d
Spca2014 building a digital workplace on share point sam marshall 1.0d
 
Machine Learning for Marketers - CTAConf 2019
Machine Learning for Marketers - CTAConf 2019Machine Learning for Marketers - CTAConf 2019
Machine Learning for Marketers - CTAConf 2019
 
Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...
Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...
Data Intelligence: How the Amalgamation of Data, Science, and Technology is C...
 
Digital Dimensions
Digital DimensionsDigital Dimensions
Digital Dimensions
 

Kürzlich hochgeladen

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Kürzlich hochgeladen (20)

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 

The Future of Search and AI

  • 1. The Future of Search & AI Trey Grainger SVP of Engineering, Lucidworks @treygrainger #Activate18 #ActivateSearch
  • 3.
  • 4. So what are the current and 
 upcoming trends in Search & AI?
  • 5.
  • 6.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13. We need big answers. Because we don’t need “big data.”
  • 14.
  • 15.
  • 16.
  • 17.
  • 18. Solr leads the way on Learning to Rank
  • 19.
  • 20. And that mass adoption of LTR looks like this..
  • 21. But what about all the other aspects of Search beyond AI?
  • 22. relevance, scale, bm25, user experience, learning to rank, autoscaling, sys admin, server provisioning, lemmatization/stemming, cloud, docker, aws, natural language processing, etl, entity extraction, tf-idf, phrase detection, semantic search, query pipelines, connectors, web crawling, signals, real-time recommendations, personalization, question answering, request handlers, search components, autocomplete, spell checking, synonyms, stop words, query-log mining, probabilistic query parsing, boolean queries, geo-spatial queries, image search, voice search, classification models, bi- directional cross-datacenter replication, semantic knowledge graphs, ontology management, merchandizing, entity type detection, query classification, word embeddings, query parsers, highlighting, analyzers, tokenization, streaming expressions, faceting, concept clustering, more like this, insight engine, apache lucene/solr, lucidworks fusion, some kind of crazy magic, that white box with the magnifying glass... What Is Search?
  • 23. There are a LOT of skillsets required to deliver a world-class Search experience
  • 24. Scaling search can be really hard
  • 25. Ease of operations, scaling, and performance are a top priority
  • 26. We're drastically improving ease of operations
  • 27. Time factors • And making the cluster aware of its own needs Size factors Budget factors • Scaling Policies • Server Metrics • Event Triggers • Disk Space Free • Prioritizing Traffic • CPU Too High • Traffic Increasing • Need more Nodes • Shards out of balance • Back Pressure • Traffic Increasing • Server Lost
  • 28. or. How I indexed *. ... and lived to tell the tale .... * simulated. 1.000.000.000.000 documents.
  • 30. There is an ongoing shift away from "big data" and toward actionable intelligence
  • 31. Assistive Search is the new paradigm •Requires personalization •Voice •Image Search •Conversational •Contextual (location, last search) •Beyond links and information to 
 Answers and Action
  • 32. ... and Search is now the de-facto user interface for 
 digital interaction with your customers
  • 33. The future of business success hinges on the outcomes of billions of digital moments FIND 
 the correct product CONNECT 
 to the right person KNOW exactly what to do next DRIVE relevant conversations RESOLVE a problem
  • 34. The Future of Search & AI Starts Now.