Full text search met ElasticSearch in de praktijk

•

0 gefällt mir•2,078 views

Jeroen van Wilgenburg

foto: Anders Sandberg
Wednesday 6 November 13

Vragen voor jullie

Wednesday 6 November 13

Vragen voor jullie
• Wie heeft Lucene wel eens gebruikt?

Wednesday 6 November 13

Vragen voor jullie
• Wie heeft Lucene wel eens gebruikt?
• SOLR?

Wednesday 6 November 13

Vragen voor jullie
• Wie heeft Lucene wel eens gebruikt?
• SOLR?
• Compass?

Wednesday 6 November 13

Vragen voor jullie
• Wie heeft Lucene wel eens gebruikt?
• SOLR?
• Compass?
• elasticsearch?

Wednesday 6 November 13

Vragen voor jullie
• Wie heeft Lucene wel eens gebruikt?
• SOLR?
• Compass?
• elasticsearch?
• waar komen de servernamen vandaan?
Wednesday 6 November 13

Wat is elasticsearch?
• full text search engine obv Lucene
• REST-api
• Distributed / High availability
• Document oriënted (nested maps)
Wednesday 6 November 13

Wie gebruiken es?

Wednesday 6 November 13

Ook elastic
• elastic compute cloud (EC2)
• elastic beanstalk
• elastic load balancer
• elastic map reduce
• elastic transcoder
• Elasticache
foto: Nick Richards
Wednesday 6 November 13

BPO-register
• vinden van koopvaardijschepen met historie
• Hibernate Envers, Spring en Twitter
Bootstrap

• begonnen met Hibernate Search, vervangen
door elasticsearch

• géén clustering gebruikt
Wednesday 6 November 13

Hibernate Search

foto: Flickr - DVIDSHUB
Wednesday 6 November 13

elasticsearch

foto: Flickr - quinn anya
Wednesday 6 November 13

Pas op met defaults!
• Gericht op ec2
• Goed om elasticsearch te leren
• 10 hits bij zoekresultaten
• voorbeeld (ﬁlmpje)
Wednesday 6 November 13

Insecure by design
• zorg dat elasticsearch in gesloten netwerk
draait

• wat er niet is kan ook niet stuk

Wednesday 6 November 13

Documentatie
• elasticsearch.org
• docs, blogs, video’s (es + youtube)
• google & stackoverﬂow
• boek van O’reilly in ontwikkeling
Wednesday 6 November 13

Community/support
• binnen dag reactie op artikel
• www.meetup.com/ElasticSearch-NL
• Gevestigd in Amsterdam

Wednesday 6 November 13

Tools
• command line curl +

chrome web inspector

• plugins
• elasticsearch head
• Inquisitor
• Elastic HQ
Wednesday 6 November 13

Query basics
• rotterdam
• name:rotterdam
• name:rotterdam +homeport:amsterdam
• +name:rotterdam +homeport:amsterdam
• *gracht
• happy abis dover
• happy abis^2 dover
Wednesday 6 November 13

Mappings
• default: dynamic mapping (schemaless)
• include_in_all

Wednesday 6 November 13

Analyzers
• index analyzer
• search analyzer
• _all is aparte analyzer (ipv per key)
• Inquisitor-plugin (demo)
Wednesday 6 November 13

Diakrieten
• instellen bij beide analyzers
• encoding van Spring
• encoding van EC2
• stackoverﬂow.com/questions/12016318/
Wednesday 6 November 13

Wildcard
• apart opgeven in query
• sufﬁx-wildcard is ok
• preﬁx is traaaaag
• boosting werkt niet meer (in onderzoek)
• “rewrite” : “scoring_boolean”
Wednesday 6 November 13

NL

AG

UK

NO

CY

LR

IT

BS

PA

BE

MT

GI

©2013 The World Flag Database & Graham Bartram

Wednesday 6 November 13

NO

IT

BE
©2013 The World Flag Database & Graham Bartram

Wednesday 6 November 13

Stop words
• a, an, and, are, as, at, be, but, by, for, if, in,

into, is, it, no, not, of, on, or, such, that, the,
their, then, there, these,they, this, to, was,
will, with

Wednesday 6 November 13

Scoring
• ‘magic’ formula
• oa. aantal voorkomens van term in hele set

Wednesday 6 November 13

Sorteren
• “Can’t sort on string types with more than

one value per doc, or more than one token
per ﬁeld”

• Voeg veld <veldnaam>.untouched toe
• http://bit.ly/wFd8CZ
Wednesday 6 November 13

Sorteren
• Lastig uitleggen aan eindgebruikers
• Gebruik een balkje met rang

Wednesday 6 November 13

Spring
• github.com/dadoonet
• Tweet David Pilato dat je het gebruikt
• Maak voldoende testqueries
• codevoorbeeld
Wednesday 6 November 13

Unit-test
• refresh na een update
• draai unit-test op andere poort/clusternaam
• codevoorbeeld

Wednesday 6 November 13

Sharding en scoring
• Default is score per shard bereken
• Cluster scores die dicht bij elkaar liggen en
ga dan sorteren

• dfs query then fetch (http://bit.ly/1cBRxMv)

Wednesday 6 November 13

Conclusie
• Snel aan de slag
• Even zoeken naar de goede documentatie
• Scoring snappen snel waardevol

Wednesday 6 November 13

Logstash + Kibana

Wednesday 6 November 13

elasticsearch in
de praktijk
Jeroen van Wilgenburg
www.jpoint.nl
@jvwilge

vanwilgenburg.wordpress.com
photo by ny156uk on Flickr

Wednesday 6 November 13

Weitere ähnliche Inhalte

Empfohlen

How to Prepare For a Successful Job Search for 2024

Albert Qian

A report by thenetworkone and Kurio. The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).

Social Media Marketing Trends 2024 // The Global Indie Insights

Kurio // The Social Media Age(ncy)

The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands. It’s important that you’re ready to implement new strategies in 2024. Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024. You’ll learn: - The latest trends in AI and automation, and what this means for an evolving paid search ecosystem. - New developments in privacy and data regulation. - Emerging ad formats that are expected to make an impact next year. Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape. If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.

Trends In Paid Search: Navigating The Digital Landscape In 2024

Search Engine Journal

From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world. With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively. The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage. Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience. See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers See the original article on Forbes here: http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b

5 Public speaking tips from TED - Visualized summary

SpeakerHub

Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world. Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance. For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age. Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.

ChatGPT and the Future of Work - Clark Boyd

Clark Boyd

Getting into the tech field. what next

Tessa Mero

Google's Just Not That Into You: Understanding Core Updates & Search Intent

Lily Ray

How to have difficult conversations

Rajiv Jayarajah, MAppComm, ACC

Introduction to Data Science

Christy Abraham Joy

Time Management & Productivity - Best Practices

Vit Horky

The six step guide to practical project management If you think managing projects is too difficult, think again. We’ve stripped back project management processes to the basics – to make it quicker and easier, without sacrificing the vital ingredients for success. “If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.” Dr Andrew Makar, Tactical Project Management

The six step guide to practical project management

MindGenius

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

RachelPearson36

During this webinar, Anand Bagmar demonstrates how AI tools such as ChatGPT can be applied to various stages of the software development life cycle (SDLC) using an eCommerce application case study. Find the on-demand recording and more info at https://applitools.info/b59 Key takeaways: • Learn how to use ChatGPT to add AI power to your testing and test automation • Understand the limitations of the technology and where human expertise is crucial • Gain insight into different AI-based tools • Adopt AI-based tools to stay relevant and optimize work for developers and testers * ChatGPT and OpenAI belong to OpenAI, L.L.C.

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

Applitools

12 Ways to Increase Your Influence at Work

GetSmarter

ChatGPT webinar slides

Alireza Esmikhani

More than Just Lines on a Map: Best Practices for U.S Bike Routes

Project for Public Spaces & National Center for Biking and Walking

Has your project been caught in a storm of deadlines, clashing requirements, and the need to change course halfway through? If yes, then check out how the administration team navigated through all of this, relocating 160 people from 3 countries and opening 2 offices during the most turbulent time in the last 20 years. Belka Games’ Chief Administrative Officer, Katerina Rudko, will share universal approaches and life hacks that can help your project survive unstable periods when there seem to be too many tasks and a lack of time and people.

Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...

DevGAMM Conference

Barbie - Brand Strategy Presentation

Erica Santiago

According to the latest State of the American Manager report from Gallup, employees who have regular meetings with their managers are almost three times as likely to be engaged as those who don’t. These regular check-ins keep managers and employees in sync and aligned. Want to see better manager/employee relationships in your organisation? Then make an all-in commitment to 1:1 meetings. Not sure how? You’ve come to the right place. In this webinar with Jamie Resker, Founder and Practice Leader for Employee Performance Solutions (EPS), and Teala Wilson, Talent Management Consultant at Saba Software, you’ll get the inside track on how to hold effective 1:1 meetings, including tips for getting managers on board. • Go beyond discussing the status of everyday work to higher level topics, including recognition, performance, development, and career aspirations • Learn how to decide meeting frequency, what to cover, as well as roles and responsibilities of the manager and employee • Understand how managers can build trust and make it comfortable for employees to provide upward feedback • Unite your organisation with a unified approach to 1:1 meetings Join us for this 1-hour webinar to get practical tips for building better manager-employee relationships with intention and purpose. About the Speakers Jamie Resker - Founder and Practice Leader for Employee Performance Solutions (EPS) Jamie Resker, Practice Leader and Founder of Employee Performance Solutions, is a recognized innovator in performance management. She is the originator of the-the Performance Continuum Feedback Method® and Conversations to Optimize Employee Performance training program; tools and training that reshape communications between managers and employees to drive and align performance. Jamie is on the faculty for the Northeast Human Resources Association, is a contributor to Halogen Software's Talent Space Blog, and is an editorial advisory board member for HR Examiner. Teala Wilson - Senior Consultant, Strategic Services, Saba Software Teala is a Talent Management Consultant at Halogen Software, now a part of Saba Software. She has worked with teams on a national and global level supporting human resources in areas such as performance management, recruitment, employee benefit programs, training and talent development, workforce planning and internal communications. Teala also has a personal passion for visual arts and design. Want to learn more? Join us for an upcoming Product Tour! http://bit.ly/2yitfqu

Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well

Saba Software

This video by Simplilearn will explain to you Introduction to C Programming Language. Introduction to C Programming Language Tutorial For Beginners will explain to you the C language's history, C's importance, its features, real-world applications, and some of its advantages and disadvantages. 00:00 Introduction to C 1:42-History of C language Dennis Ritchie, a computer scientist, could identify the gaps and tap out the best features from both B and BCPL languages to invent a new hybrid. Hence, C was born in 1972 at Bell Laboratories. A remarkably simple and highly readable programming language resulted in groundbreaking advancements in the IT industry. 2:48-Importance and unraveling the powerful capabilities of C, The widespread use of C started to take over the IT industry. Unraveling the potential of C, the designers began to discover new possibilities that led them to focus on the big picture. 3:56-C's cutting-edge features The designers at Bell Laboratories ensured that their programming language solved the issues with B and BCPL and the ones they had foreseen. 6:35-The popular real-world applications of C -UNIX operating system -google file system -Mozilla -Graphical user interface 8:30-The advantages and disadvantages of C 10:34-The popular IT companies and their domains that employ C · MasterCard · IBM · Flipkart · Dell · Twitter · GitHub and twitch 11:09-First c program. 🔥 Explore our FREE courses with completion certificates: https://www.simplilearn.com/skillup-f... ✅Subscribe to our Channel to learn more about the top Technologies: https://bit.ly/2VT4WtH ⏩ Check out the C++ Programming training videos: https://www.youtube.com/playlist?list... #IntroductiontoCProgrammingLanguage #CLanguage #CProgramming #CProgram #CProgrammingLanguage #LearnCProgramming #HowToCodeInCForBeginners #CTutorialForBeginners #LearnCProgramming #Simplilearn Dennis Ritchie, a computer scientist, was able to identify the gaps and tap out the best features from both B and BCPL languages to invent a new hybrid. Hence, C was born in 1972 at Bell Laboratories. A remarkably simple and highly readable programming language resulted in groundbreaking advancements in the IT industry. ✅What is C++ Programming? C++ is an enhanced and extended version of C programming language, developed by Bjarne Stroustrup in 1979 as part of his Ph.D. project. Bjarne developed what he called ‘C with Classes’ (later renamed C++) because he felt limited by the existing programming languages that were not ideal for large scale projects. He used C to build what he wanted because C was already a general-purpose language that was efficient and fast in its operations. ✅C++ Career Prospects: With just C++ programming expertise, you will have excellent job opportunities, salaries, and career prospects. However, for a career based on programming languages such as Java and Python (which are in more demand than C++) or for careers based on front-end, back-end, and full-stack

Introduction to C Programming Language

Simplilearn

Empfohlen (20)

How to Prepare For a Successful Job Search for 2024

Social Media Marketing Trends 2024 // The Global Indie Insights

Trends In Paid Search: Navigating The Digital Landscape In 2024

5 Public speaking tips from TED - Visualized summary

ChatGPT and the Future of Work - Clark Boyd

Getting into the tech field. what next

Google's Just Not That Into You: Understanding Core Updates & Search Intent

How to have difficult conversations

Introduction to Data Science

Time Management & Productivity - Best Practices

The six step guide to practical project management

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

12 Ways to Increase Your Influence at Work

ChatGPT webinar slides

More than Just Lines on a Map: Best Practices for U.S Bike Routes

Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...

Barbie - Brand Strategy Presentation

Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well

Introduction to C Programming Language

Full text search met ElasticSearch in de praktijk

1. foto: Anders Sandberg Wednesday 6 November 13

2. Full text search met elasticsearch in de praktijk Jeroen van Wilgenburg, JPoint @jvwilge vanwilgenburg.wordpress.com Wednesday 6 November 13

3. Vragen voor jullie Wednesday 6 November 13

4. Vragen voor jullie • Wie heeft Lucene wel eens gebruikt? Wednesday 6 November 13

5. Vragen voor jullie • Wie heeft Lucene wel eens gebruikt? • SOLR? Wednesday 6 November 13

6. Vragen voor jullie • Wie heeft Lucene wel eens gebruikt? • SOLR? • Compass? Wednesday 6 November 13

7. Vragen voor jullie • Wie heeft Lucene wel eens gebruikt? • SOLR? • Compass? • elasticsearch? Wednesday 6 November 13

8. Vragen voor jullie • Wie heeft Lucene wel eens gebruikt? • SOLR? • Compass? • elasticsearch? • waar komen de servernamen vandaan? Wednesday 6 November 13

9. Wat is elasticsearch? • full text search engine obv Lucene • REST-api • Distributed / High availability • Document oriënted (nested maps) Wednesday 6 November 13

10. Wie gebruiken es? Wednesday 6 November 13

11. Ook elastic • elastic compute cloud (EC2) • elastic beanstalk • elastic load balancer • elastic map reduce • elastic transcoder • Elasticache foto: Nick Richards Wednesday 6 November 13

12. BPO-register • vinden van koopvaardijschepen met historie • Hibernate Envers, Spring en Twitter Bootstrap • begonnen met Hibernate Search, vervangen door elasticsearch • géén clustering gebruikt Wednesday 6 November 13

13. Hibernate Search foto: Flickr - DVIDSHUB Wednesday 6 November 13

14. elasticsearch foto: Flickr - quinn anya Wednesday 6 November 13

15. Clusternaam Wednesday 6 November 13

16. Pas op met defaults! • Gericht op ec2 • Goed om elasticsearch te leren • 10 hits bij zoekresultaten • voorbeeld (ﬁlmpje) Wednesday 6 November 13

17. Insecure by design • zorg dat elasticsearch in gesloten netwerk draait • wat er niet is kan ook niet stuk Wednesday 6 November 13

18. Documentatie • elasticsearch.org • docs, blogs, video’s (es + youtube) • google & stackoverﬂow • boek van O’reilly in ontwikkeling Wednesday 6 November 13

19. Community/support • binnen dag reactie op artikel • www.meetup.com/ElasticSearch-NL • Gevestigd in Amsterdam Wednesday 6 November 13

20. Tools • command line curl + chrome web inspector • plugins • elasticsearch head • Inquisitor • Elastic HQ Wednesday 6 November 13

21. Query basics • rotterdam • name:rotterdam • name:rotterdam +homeport:amsterdam • +name:rotterdam +homeport:amsterdam • *gracht • happy abis dover • happy abis^2 dover Wednesday 6 November 13

22. Mappings • default: dynamic mapping (schemaless) • include_in_all Wednesday 6 November 13

23. Analyzers • index analyzer • search analyzer • _all is aparte analyzer (ipv per key) • Inquisitor-plugin (demo) Wednesday 6 November 13

24. Diakrieten • instellen bij beide analyzers • encoding van Spring • encoding van EC2 • stackoverﬂow.com/questions/12016318/ Wednesday 6 November 13

25. Wildcard • apart opgeven in query • sufﬁx-wildcard is ok • preﬁx is traaaaag • boosting werkt niet meer (in onderzoek) • “rewrite” : “scoring_boolean” Wednesday 6 November 13

29. Stop words • a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, there, these,they, this, to, was, will, with Wednesday 6 November 13

30. Scoring • ‘magic’ formula • oa. aantal voorkomens van term in hele set Wednesday 6 November 13

31. Sorteren • “Can’t sort on string types with more than one value per doc, or more than one token per ﬁeld” • Voeg veld <veldnaam>.untouched toe • http://bit.ly/wFd8CZ Wednesday 6 November 13

32. Sorteren • Lastig uitleggen aan eindgebruikers • Gebruik een balkje met rang Wednesday 6 November 13

33. Spring • github.com/dadoonet • Tweet David Pilato dat je het gebruikt • Maak voldoende testqueries • codevoorbeeld Wednesday 6 November 13

34. Unit-test • refresh na een update • draai unit-test op andere poort/clusternaam • codevoorbeeld Wednesday 6 November 13

35. Sharding en scoring • Default is score per shard bereken • Cluster scores die dicht bij elkaar liggen en ga dan sorteren • dfs query then fetch (http://bit.ly/1cBRxMv) Wednesday 6 November 13

36. Conclusie • Snel aan de slag • Even zoeken naar de goede documentatie • Scoring snappen snel waardevol Wednesday 6 November 13

37. Vragen? Wednesday 6 November 13

38. Logstash + Kibana Wednesday 6 November 13

39. elasticsearch in de praktijk Jeroen van Wilgenburg www.jpoint.nl @jvwilge vanwilgenburg.wordpress.com photo by ny156uk on Flickr Wednesday 6 November 13