SlideShare ist ein Scribd-Unternehmen logo
1 von 37
Downloaden Sie, um offline zu lesen
Experiments with AI and
OpenSearch
Jochen Kressin, Eliatra
Leanne Lacey-Byrne, Eliatra
About Eliatra
Founded 2020 by the makers of Search Guard
Focus on OpenSearch
Official OpenSearch Partner
Contributor to Security Plugin
OpenSearch Development, Support, Consulting
and Training
Why this talk?
Obvious: LLMs and accompanying technologies
Hyped – or Overhyped?
LLMs and AI are here to stay
They will affect the way we work
Two experiments
Use LLMs to make technology accessible
Auto-generate OpenSearch queries
Leanne Lacey-Byrne
What’s the fuzz about Vector Search and RAG?
Explore Vector capabilities of OpenSearch
Jochen Kressin
Problem Statement
ChatGPT could make OpenSearch more accessible.
OpenSearch is queried via Domain Specific Language (DSL).
Queries can become complex - large JSON objects.
Traditionally, you couldn't use SQL to query OpenSearch.
Need someone proficient with this technology to do the queries.
Set up for Investigation
Given there is more public data on Elasticsearch, the investigation was
started here
Added sample data sets:
kibana_sample_data_flights,
kibana_sample_data_logs
kibana_sample_data_ecommerce
Key Screens for Investigation
https://localhost:5601/app/dashboards#/view/7adfa750-4c81-11e8-b3d7-01146121b73d?_g=(filters:!())
https://localhost:5601/app/dashboards
https://localhost:5601/app/management/data/index_management/indices
https://localhost:5601/app/dev_tools#/console
Elasticsearch/OpenSearch Mappings
“Mapping is the process of defining how a document, and the fields it
contains, are stored and indexed. Each document is a collection of fields,
which each have their own data type. When mapping your data, you
create a mapping definition, which contains a list of fields that are
pertinent to the document."
Mapping API
Get the mapping of an index:
curl -X GET "localhost:9200/kibana_sample_data_logs/
_mapping"
With Security enabled:
curl -u admin:admin -k -X GET "https://localhost:9200/
kibana_sample_data_logs/_mapping"
(Add –k flag to add the self-signed certificates)
Mapping API
curl -k -u admin:admin -XGET https://localhost:9200/kibana_sample_data_logs/_mapping
".ds-kibana_sample_data_logs-2023.11.08-000001" : {
"mappings" : {
...
"properties" : {
"@timestamp" : {
"type" : "date"
},
"agent" : {
"type" : "text",
...
},
"bytes" : {
"type" : "long"
},
...
"clientip" : {
"type" : "ip"
},
"host" : {
"type" : "text",
...
}
...
}
Proposal
With the map of the indices, we can give this to ChatGPT and ask it
to generate queries for us i.e.
You are a data scientist that works with Elasticsearch.
You data is located in an index <indexname> and the
data mapping looks like:
<add mapping here>
Generate an Elasticsearch query to answer the following
question:
<question in natural language>
ChatGPTs Query Construction
Question: Please give me the total number of website visits in the last week
Phase 1:
ChatGPT 3.5, ChatGPT 4.0
Phase 2:
ChatGPT 3.5, ChatGPT 4.0, Bard, OpsGPT
Phase 3:
Enhancing the performance
Results for ChatGPT
Phase 1
Dashboard ChatGPT 3.5 +
Mapping
(correct %)
Chat GPT 3.5 +
Mapping +
Sampling
(correct %)
Chat GPT 4.0 +
Mapping
(correct %)
Chat GPT 4.0 +
Mapping +
Sampling
(correct %)
Global Flight
Dashboard
10% 30% 0% 16%
Website 50% 25% 0% 0%
E-commerce 20% 60% 0% 0%
Overall 27 % 33 % 0 % 5 %
Results for Chat, Bard & OpsGpt
Phase 2
Dashboard Chat GPT 3.5 +
Mapping +
Sampling
(correct %)
Bard OpsGPT
Overall 33 % 13 % 27 %
Phase 3 - Improving results
Continue exploring new avenues to continue testing the LLM feasibility
Best results obtained: 33% (ChatGPT 3.5 + Mapping + Sample Data)
Ideally we get a hit rate of > 70% to look at developing the solution further in
code. So far, this was all done with no code, just running the queries
generated by the LLM in the Elasticsearch console as a proof of concept.
Next step is providing more info to the LLM interface to get better results.
Fine-tune ChatGPT
From ChatGPT docs:
Fine-tuning lets you get more out of the models available through
the API by providing:
• Higher quality results than prompting
• Ability to train on more examples than can fit in a prompt
• Token savings due to shorter prompts
• Lower latency requests
Results
• Too small sample size
Keyword Search
Still predominant model
Based on inverted index
“take words from a query, look for match in index”
Efficient, proven, effective
Exact/fuzzy matching, multi-field/boolean searches
Ranking (BM25)
Facets and Snippets
But: No Context!
Semantic Search
„Vector Search“
Attempt to understand
Context
Meaning
Intent
Concepts
Transform sources to vectors
Text, images, audio
High-dimensional space
Compute similarity between document and query vector
Semantic Search
Source: https://blog.det.life/vector-database-concepts-and-examples-f73d7e683d3e
Semantic Search: Example
Dataset of public images with captions
Search for „Wild West“
Expectation:
Words with similar context/meaning are closer to each
other in the vector space
Search results returned by context, not just terms
Semantic Search: Example
Semantic search vs. Keyword Search
OpenSearch: Neural Search Plugin
Available since OpenSearch 2.4
Text-to-Vector
Ingest Time
Query Time
Supports multiple models
Pre-trained (Huggingface) / Custom
K-NN (exact and approximate)
Apache2 license
Neural Search Plugin
https://opensearch.org/docs/latest/ml-commons-plugin/semantic-search/
Neural Search Plugin
https://opensearch.org/docs/latest/ml-commons-plugin/semantic-search/
Pre-trained Models
https://opensearch.org/docs/latest/ml-commons-plugin/pretrained-models/
Neural Search Plugin: Basic Steps
Set up an ML language model
Choose a language model
Register a model group, register model to group
Deploy the model
Ingest data with neural search.
Create an ingest pipeline for neural search.
Create a k-NN index.
Ingest documents into the index.
Search the data.
Search using a keyword, neural, or hybrid search.
Neural Search Plugin: Set up model
POST /_plugins/_ml/model_groups/_register
{
"name": "NLP_model_group",
"description": "A model group for NLP models",
"access_mode": "public"
}
POST /_plugins/_ml/models/_register
{
"name": "huggingface/sentence-transformers/msmarco-distilbert-base-tas-b",
"version": "1.0.1",
"model_group_id": "Z1eQf4oB5Vm0Tdw8EIP2",
"model_format": "TORCH_SCRIPT"
}
POST /_plugins/_ml/models/aVeif4oB5Vm0Tdw8zYO2/_deploy
Neural Search Plugin: Create Ingest Pipeline
PUT /_ingest/pipeline/nlp-ingest-pipeline
{
"description": "An NLP ingest pipeline",
"processors": [
{
"text_embedding": {
"model_id": "aVeif4oB5Vm0Tdw8zYO2",
"field_map": {
"text": "passage_embedding"
}
}
}
]
}
Neural Search Plugin: Create Index
PUT /my-nlp-index
{
"settings": {
"index.knn": true,
"default_pipeline": "nlp-ingest-pipeline"
},
"mappings": {
"properties": {
"id": { "type": "text” },
"passage_embedding": {
"type": "knn_vector",
"dimension": 768,
…
},
"text": {
"type": "text"
}
}
}
}
Neural Search Plugin: Indexing and Searching
Quick Demo
Semantic Search Learnings
Interpretability: Why are these documents returned?
Results depend highly on the used model
Ranking/Scoring difficult to optimize
Training/tuning for domain-specific tasks
Performance can be an issue
Hybrid search promising for search tasks
Both models will co-exist
Retrieval Augmented Generation (RAG)
LLMs are static, knowledge “frozen in time”
General-purpose, lack of domain-specific knowledge
Tend to hallucinate
LLMs will always give you an answer, even incorrect
Black box: ”Where does this answer come from?”
Fine-tuning?
Needs good training data
Not possible on-the-fly
New knowledge becomes part of the model
Retrieval Augmented Generation (RAG)
Retrieval Augmented Generation (RAG)
Give LLMs access to information beyond training data
Published in a 2020 paper by Meta
Combine LLM with external data sources
For example: Vector Database ;)
Add contextual information on prompt-level
Retrieval Augmented Generation (RAG)
What about logs?
Research Papers (2023)
“Log Parsing: How Far Can ChatGPT Go?”
“Log Exploration and Analytics Using Large Language Models”
“Review of popular word embedding models for event log anomaly detection purposes”
“LogEvent-to-Vector Based Anomaly Detection for Large-Scale Logs in Internet of Things”
Offerings (2023)
ChaosSearch
Structured

Weitere ähnliche Inhalte

Ähnlich wie OSMC 2023 | Experiments with OpenSearch and AI by Jochen Kressin & Leanne Lace-Byrne

Data Science Challenge presentation given to the CinBITools Meetup Group
Data Science Challenge presentation given to the CinBITools Meetup GroupData Science Challenge presentation given to the CinBITools Meetup Group
Data Science Challenge presentation given to the CinBITools Meetup Group
Doug Needham
 

Ähnlich wie OSMC 2023 | Experiments with OpenSearch and AI by Jochen Kressin & Leanne Lace-Byrne (20)

Pinterest - Big Data Machine Learning Platform at Pinterest
Pinterest - Big Data Machine Learning Platform at PinterestPinterest - Big Data Machine Learning Platform at Pinterest
Pinterest - Big Data Machine Learning Platform at Pinterest
 
Optimizing GenAI apps, by N. El Mawass and Maria Knorps
Optimizing GenAI apps, by N. El Mawass and Maria KnorpsOptimizing GenAI apps, by N. El Mawass and Maria Knorps
Optimizing GenAI apps, by N. El Mawass and Maria Knorps
 
Data Science Challenge presentation given to the CinBITools Meetup Group
Data Science Challenge presentation given to the CinBITools Meetup GroupData Science Challenge presentation given to the CinBITools Meetup Group
Data Science Challenge presentation given to the CinBITools Meetup Group
 
Cloudera Data Science Challenge
Cloudera Data Science ChallengeCloudera Data Science Challenge
Cloudera Data Science Challenge
 
Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14
 
How Azure helps to build better business processes and customer experiences w...
How Azure helps to build better business processes and customer experiences w...How Azure helps to build better business processes and customer experiences w...
How Azure helps to build better business processes and customer experiences w...
 
Generative AI in CSharp with Semantic Kernel.pptx
Generative AI in CSharp with Semantic Kernel.pptxGenerative AI in CSharp with Semantic Kernel.pptx
Generative AI in CSharp with Semantic Kernel.pptx
 
Story line
Story lineStory line
Story line
 
Use Case Patterns for LLM Applications (1).pdf
Use Case Patterns for LLM Applications (1).pdfUse Case Patterns for LLM Applications (1).pdf
Use Case Patterns for LLM Applications (1).pdf
 
DevOps for DataScience
DevOps for DataScienceDevOps for DataScience
DevOps for DataScience
 
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
Microsoft Build 2023 Updates – Copilot Stack and Azure OpenAI Service (Machin...
 
Learning to rank search results
Learning to rank search resultsLearning to rank search results
Learning to rank search results
 
Sumo Logic Cert Jam - Security & Compliance
Sumo Logic Cert Jam - Security & ComplianceSumo Logic Cert Jam - Security & Compliance
Sumo Logic Cert Jam - Security & Compliance
 
B4UConference_machine learning_deeplearning
B4UConference_machine learning_deeplearningB4UConference_machine learning_deeplearning
B4UConference_machine learning_deeplearning
 
clicks2conversations.pdf
clicks2conversations.pdfclicks2conversations.pdf
clicks2conversations.pdf
 
[2 d1] elasticsearch 성능 최적화
[2 d1] elasticsearch 성능 최적화[2 d1] elasticsearch 성능 최적화
[2 d1] elasticsearch 성능 최적화
 
Are API Services Taking Over All the Interesting Data Science Problems?
Are API Services Taking Over All the Interesting Data Science Problems?Are API Services Taking Over All the Interesting Data Science Problems?
Are API Services Taking Over All the Interesting Data Science Problems?
 
Building Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startBuilding Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to start
 
Serverless Machine Learning
Serverless Machine LearningServerless Machine Learning
Serverless Machine Learning
 
[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화[2D1]Elasticsearch 성능 최적화
[2D1]Elasticsearch 성능 최적화
 

Kürzlich hochgeladen

Kürzlich hochgeladen (10)

Breathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptxBreathing in New Life_ Part 3 05 22 2024.pptx
Breathing in New Life_ Part 3 05 22 2024.pptx
 
Microsoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdf
Microsoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdfMicrosoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdf
Microsoft Fabric Analytics Engineer (DP-600) Exam Dumps 2024.pdf
 
2024-05-15-Surat Meetup-Hyperautomation.pptx
2024-05-15-Surat Meetup-Hyperautomation.pptx2024-05-15-Surat Meetup-Hyperautomation.pptx
2024-05-15-Surat Meetup-Hyperautomation.pptx
 
Understanding Poverty: A Community Questionnaire
Understanding Poverty: A Community QuestionnaireUnderstanding Poverty: A Community Questionnaire
Understanding Poverty: A Community Questionnaire
 
Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.Deciding The Topic of our Magazine.pptx.
Deciding The Topic of our Magazine.pptx.
 
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdfACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
ACM CHT Best Inspection Practices Kinben Innovation MIC Slideshare.pdf
 
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docxThe Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
The Influence and Evolution of Mogul Press in Contemporary Public Relations.docx
 
Databricks Machine Learning Associate Exam Dumps 2024.pdf
Databricks Machine Learning Associate Exam Dumps 2024.pdfDatabricks Machine Learning Associate Exam Dumps 2024.pdf
Databricks Machine Learning Associate Exam Dumps 2024.pdf
 
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptxDAY 0 8 A Revelation 05-19-2024 PPT.pptx
DAY 0 8 A Revelation 05-19-2024 PPT.pptx
 
ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024ServiceNow CIS-Discovery Exam Dumps 2024
ServiceNow CIS-Discovery Exam Dumps 2024
 

OSMC 2023 | Experiments with OpenSearch and AI by Jochen Kressin & Leanne Lace-Byrne

  • 1. Experiments with AI and OpenSearch Jochen Kressin, Eliatra Leanne Lacey-Byrne, Eliatra
  • 2. About Eliatra Founded 2020 by the makers of Search Guard Focus on OpenSearch Official OpenSearch Partner Contributor to Security Plugin OpenSearch Development, Support, Consulting and Training
  • 3. Why this talk? Obvious: LLMs and accompanying technologies Hyped – or Overhyped? LLMs and AI are here to stay They will affect the way we work
  • 4. Two experiments Use LLMs to make technology accessible Auto-generate OpenSearch queries Leanne Lacey-Byrne What’s the fuzz about Vector Search and RAG? Explore Vector capabilities of OpenSearch Jochen Kressin
  • 5. Problem Statement ChatGPT could make OpenSearch more accessible. OpenSearch is queried via Domain Specific Language (DSL). Queries can become complex - large JSON objects. Traditionally, you couldn't use SQL to query OpenSearch. Need someone proficient with this technology to do the queries.
  • 6. Set up for Investigation Given there is more public data on Elasticsearch, the investigation was started here Added sample data sets: kibana_sample_data_flights, kibana_sample_data_logs kibana_sample_data_ecommerce
  • 7. Key Screens for Investigation https://localhost:5601/app/dashboards#/view/7adfa750-4c81-11e8-b3d7-01146121b73d?_g=(filters:!()) https://localhost:5601/app/dashboards https://localhost:5601/app/management/data/index_management/indices https://localhost:5601/app/dev_tools#/console
  • 8. Elasticsearch/OpenSearch Mappings “Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. Each document is a collection of fields, which each have their own data type. When mapping your data, you create a mapping definition, which contains a list of fields that are pertinent to the document."
  • 9. Mapping API Get the mapping of an index: curl -X GET "localhost:9200/kibana_sample_data_logs/ _mapping" With Security enabled: curl -u admin:admin -k -X GET "https://localhost:9200/ kibana_sample_data_logs/_mapping" (Add –k flag to add the self-signed certificates)
  • 10. Mapping API curl -k -u admin:admin -XGET https://localhost:9200/kibana_sample_data_logs/_mapping ".ds-kibana_sample_data_logs-2023.11.08-000001" : { "mappings" : { ... "properties" : { "@timestamp" : { "type" : "date" }, "agent" : { "type" : "text", ... }, "bytes" : { "type" : "long" }, ... "clientip" : { "type" : "ip" }, "host" : { "type" : "text", ... } ... }
  • 11. Proposal With the map of the indices, we can give this to ChatGPT and ask it to generate queries for us i.e. You are a data scientist that works with Elasticsearch. You data is located in an index <indexname> and the data mapping looks like: <add mapping here> Generate an Elasticsearch query to answer the following question: <question in natural language>
  • 12. ChatGPTs Query Construction Question: Please give me the total number of website visits in the last week
  • 13. Phase 1: ChatGPT 3.5, ChatGPT 4.0 Phase 2: ChatGPT 3.5, ChatGPT 4.0, Bard, OpsGPT Phase 3: Enhancing the performance
  • 14. Results for ChatGPT Phase 1 Dashboard ChatGPT 3.5 + Mapping (correct %) Chat GPT 3.5 + Mapping + Sampling (correct %) Chat GPT 4.0 + Mapping (correct %) Chat GPT 4.0 + Mapping + Sampling (correct %) Global Flight Dashboard 10% 30% 0% 16% Website 50% 25% 0% 0% E-commerce 20% 60% 0% 0% Overall 27 % 33 % 0 % 5 %
  • 15. Results for Chat, Bard & OpsGpt Phase 2 Dashboard Chat GPT 3.5 + Mapping + Sampling (correct %) Bard OpsGPT Overall 33 % 13 % 27 %
  • 16. Phase 3 - Improving results Continue exploring new avenues to continue testing the LLM feasibility Best results obtained: 33% (ChatGPT 3.5 + Mapping + Sample Data) Ideally we get a hit rate of > 70% to look at developing the solution further in code. So far, this was all done with no code, just running the queries generated by the LLM in the Elasticsearch console as a proof of concept. Next step is providing more info to the LLM interface to get better results.
  • 17. Fine-tune ChatGPT From ChatGPT docs: Fine-tuning lets you get more out of the models available through the API by providing: • Higher quality results than prompting • Ability to train on more examples than can fit in a prompt • Token savings due to shorter prompts • Lower latency requests Results • Too small sample size
  • 18. Keyword Search Still predominant model Based on inverted index “take words from a query, look for match in index” Efficient, proven, effective Exact/fuzzy matching, multi-field/boolean searches Ranking (BM25) Facets and Snippets But: No Context!
  • 19. Semantic Search „Vector Search“ Attempt to understand Context Meaning Intent Concepts Transform sources to vectors Text, images, audio High-dimensional space Compute similarity between document and query vector
  • 21. Semantic Search: Example Dataset of public images with captions Search for „Wild West“ Expectation: Words with similar context/meaning are closer to each other in the vector space Search results returned by context, not just terms
  • 23. Semantic search vs. Keyword Search
  • 24. OpenSearch: Neural Search Plugin Available since OpenSearch 2.4 Text-to-Vector Ingest Time Query Time Supports multiple models Pre-trained (Huggingface) / Custom K-NN (exact and approximate) Apache2 license
  • 28. Neural Search Plugin: Basic Steps Set up an ML language model Choose a language model Register a model group, register model to group Deploy the model Ingest data with neural search. Create an ingest pipeline for neural search. Create a k-NN index. Ingest documents into the index. Search the data. Search using a keyword, neural, or hybrid search.
  • 29. Neural Search Plugin: Set up model POST /_plugins/_ml/model_groups/_register { "name": "NLP_model_group", "description": "A model group for NLP models", "access_mode": "public" } POST /_plugins/_ml/models/_register { "name": "huggingface/sentence-transformers/msmarco-distilbert-base-tas-b", "version": "1.0.1", "model_group_id": "Z1eQf4oB5Vm0Tdw8EIP2", "model_format": "TORCH_SCRIPT" } POST /_plugins/_ml/models/aVeif4oB5Vm0Tdw8zYO2/_deploy
  • 30. Neural Search Plugin: Create Ingest Pipeline PUT /_ingest/pipeline/nlp-ingest-pipeline { "description": "An NLP ingest pipeline", "processors": [ { "text_embedding": { "model_id": "aVeif4oB5Vm0Tdw8zYO2", "field_map": { "text": "passage_embedding" } } } ] }
  • 31. Neural Search Plugin: Create Index PUT /my-nlp-index { "settings": { "index.knn": true, "default_pipeline": "nlp-ingest-pipeline" }, "mappings": { "properties": { "id": { "type": "text” }, "passage_embedding": { "type": "knn_vector", "dimension": 768, … }, "text": { "type": "text" } } } }
  • 32. Neural Search Plugin: Indexing and Searching Quick Demo
  • 33. Semantic Search Learnings Interpretability: Why are these documents returned? Results depend highly on the used model Ranking/Scoring difficult to optimize Training/tuning for domain-specific tasks Performance can be an issue Hybrid search promising for search tasks Both models will co-exist
  • 34. Retrieval Augmented Generation (RAG) LLMs are static, knowledge “frozen in time” General-purpose, lack of domain-specific knowledge Tend to hallucinate LLMs will always give you an answer, even incorrect Black box: ”Where does this answer come from?” Fine-tuning? Needs good training data Not possible on-the-fly New knowledge becomes part of the model
  • 35. Retrieval Augmented Generation (RAG) Retrieval Augmented Generation (RAG) Give LLMs access to information beyond training data Published in a 2020 paper by Meta Combine LLM with external data sources For example: Vector Database ;) Add contextual information on prompt-level
  • 37. What about logs? Research Papers (2023) “Log Parsing: How Far Can ChatGPT Go?” “Log Exploration and Analytics Using Large Language Models” “Review of popular word embedding models for event log anomaly detection purposes” “LogEvent-to-Vector Based Anomaly Detection for Large-Scale Logs in Internet of Things” Offerings (2023) ChaosSearch Structured