Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Clinton Gormley – Elasticsearch Query DSL – Not just for wizards…- NoSQL matters Barcelona 2014

Clinton Gormley – Elasticsearch Query DSL – Not just for wizards…

The Elasticsearch Query DSL is a rich, flexible, powerful query language for full text and structured search, but with power comes complexity. Which of the 40 available queries should you use? What's a filter and when should you use it? How do you combine multiple filters, or multiple queries or queries with filters?To most users, "relevance", and how it is affected by different queries, is a black box. Multi-field queries in particular can be difficult to get right if you don't understand how they work. In this talk, I will explain the Query DSL from the ground up: how filters and queries use the inverted index to find matching documents, how the relevance score is calculated, how to combine the filter/query building blocks into complex statements. And finally, I will talk about the pitfalls of multi-field queries and how to avoid them.

  • Als Erste(r) kommentieren

Clinton Gormley – Elasticsearch Query DSL – Not just for wizards…- NoSQL matters Barcelona 2014

  1. 1. Elasticsearch Query DSL … not just for wizards Clinton Gormley @clintongormley Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  2. 2. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  3. 3. elasticsearch.org/guide Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  4. 4. elasticsearch Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  5. 5. elasticsearch • real-time Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  6. 6. elasticsearch • real-time • distributed Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  7. 7. elasticsearch • real-time • distributed • search Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  8. 8. elasticsearch • real-time • distributed • search • analytics Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  9. 9. mapping analysis query dsl Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  10. 10. mapping analysis query dsl Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  11. 11. query dsl Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  12. 12. query dsl flexible, powerful query language Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  13. 13. queries • relevance • full text • not cached • slower filters • boolean yes/no • exact values • cached • faster Filter first, then query remaining docs Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  14. 14. GET /_search Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  15. 15. GET /_search { "query": {...} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  16. 16. GET /_search { "query": { "match": { "title": "search" }} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  17. 17. GET /_search { "query": { match_all: {}} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  18. 18. GET /_search { "query": { match_all: {}} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  19. 19. GET /_search { "query": { "filtered": { "query": {...}, "filter": {...} } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  20. 20. GET /_search { "query": { "filtered": { "query": { "match": { "title": "search" }}, "filter": {...} } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  21. 21. GET /_search { "query": { "filtered": { "query": { "match": { "title": "search" }}, "filter": { "term": { "status": "active" }} } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  22. 22. GET /_search { "query": { "filtered": { "query": { "match_all": {}}, "filter": { "term": { "status": "active" }} } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  23. 23. GET /_search { "query": { "filtered": { "query": { "match_all": {}}, "filter": { "term": { "status": "active" }} } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  24. 24. how data is indexed Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  25. 25. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  26. 26. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  27. 27. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } where content like “%brown%fox%” { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  28. 28. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } slow & inflexible { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  29. 29. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } “analysis” { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  30. 30. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  31. 31. { "title": [quick,brown,rabbits], "content": [brown,rabbits,are,commonly,seen] } { "title": [keeping,pets,healthy], "content": [my,quick,brown,fox,eats,rabbits,on,a, regular,basis] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  32. 32. field: content Term Doc 1 Doc 2 a are basis brown commonly eats fox my on quick rabbits regular seen sorted list of unique terms Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  33. 33. field: content Term Doc 1 Doc 2 a are basis brown commonly eats fox my on quick rabbits regular seen where they occur Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  34. 34. field: content Term Doc 1 Doc 2 a are basis brown commonly eats fox my on quick rabbits regular seen Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  35. 35. field: content Term Doc 1 Doc 2 a are basis brown commonly eats fox my on quick rabbits regular seen Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  36. 36. inverted index Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  37. 37. inverted index not just for text Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  38. 38. inverted index numbers, dates, bools, enums geopoints, geoshapes, etc Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  39. 39. WHERE field = "value" Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  40. 40. WHERE field CONTAINS "value" Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  41. 41. WHERE field CONTAINS "value" term filter Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  42. 42. WHERE field CONTAINS "value" "term": { "title": "brown" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  43. 43. GET /_search { "query": { } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  44. 44. GET /_search { "query": { "filtered": { } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  45. 45. GET /_search { "query": { "filtered": { "query": { ... }, "filter": { ... } } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  46. 46. GET /_search { "query": { "filtered": { "query": { "match_all": {} }, "filter": { ... } } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  47. 47. GET /_search { "query": { "filtered": { "query": { "match_all": {} }, "filter": { ... } } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  48. 48. GET /_search { "query": { "filtered": { "query": { "match_all": {} }, "filter": { "term": { "title": "brown" }} } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  49. 49. field: title Term Doc 1 Doc 2 brown healthy keeping pets quick rabbits Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  50. 50. field: title Term Doc 1 Doc 2 brown healthy keeping pets quick rabbits Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  51. 51. "term": { "title": "brown" } ➔ result: bitset[ 1, 0 ] ➔ cache as: "title:brown" Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  52. 52. WHERE field IN ["val",…] Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  53. 53. WHERE field IN ["val",…] terms filter Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  54. 54. WHERE field IN ["val",…] "terms": { "title": ["quick", "pets"] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  55. 55. field: title Term Doc 1 Doc 2 brown healthy keeping pets quick rabbits Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  56. 56. "terms": { "title": ["quick","pets"] } ➔ result: bitset[ 1, 1 ] ➔ cache as: "title:quick title:pets" Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  57. 57. WHERE field >= "val1" AND field < "val2" Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  58. 58. WHERE field >= "val1" AND field < "val2" range filter Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  59. 59. WHERE field >= "val1" AND field < "val2" "range": { "content":{ "gte": "a", "lt": "m" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  60. 60. field: content Term Doc 1 Doc 2 a are basis brown commonly eats fox my on quick rabbits regular seen Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  61. 61. "range": { "content":{ "gte": "a", "lt": "m" } ➔ result: bitset[ 1, 1 ] ➔ cache as: "content:[a TO m}" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  62. 62. "range": { "date":{ "gte": "2014-­‐01-­‐01", "lt": "2041-­‐02-­‐01" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  63. 63. "range": { numeric/date fields "date":{ "gte": "2014-­‐01-­‐01", "lt": "2041-­‐02-­‐01" } } optimised for range filters Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  64. 64. "range": { "date":{ "gte": "now -­‐ 1h" } } not cached Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  65. 65. "range": { "date":{ "gte": "now -­‐ 1h / h" } } cached Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  66. 66. WHERE field IS NOT NULL Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  67. 67. WHERE field has any term Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  68. 68. WHERE field has any term exists filter Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  69. 69. WHERE field has any term "exists": { "field": "title" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  70. 70. WHERE field has no term missing filter Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  71. 71. WHERE field has no term "missing": { "field": "title" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  72. 72. AND … OR … NOT Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  73. 73. AND … OR … NOT bool filter Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  74. 74. "bool": { "must": [ <filters> ], "should": [ <filters> ], "must_not": [ <filters> ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  75. 75. "bool": { "must": [ <filters> ], # AND "should": [ <filters> ], "must_not": [ <filters> ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  76. 76. "bool": { "must": [ <filters> ], "should": [ <filters> ], # OR "must_not": [ <filters> ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  77. 77. "bool": { "must": [ <filters> ], "should": [ <filters> ], "must_not": [ <filters> ] # NOT } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  78. 78. "bool": { "must": { "term": { "title": "rabbits" }}, } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  79. 79. "bool": { "must": { "term": { "title": "rabbits" }}, "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ], } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  80. 80. "bool": { "must": { "term": { "title": "rabbits" }}, "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ], "must_not": { "term": { "content": "fox" }} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  81. 81. "bool": { "must": [ { "bool": { "should": [ { "term": { "title": "rabbits" }}, { "term": { "content": "rabbits" }} ]}}, { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ]}} ], "must_not": { "term": { "content": "fox" }} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  82. 82. "bool": { "must": [ { "bool": { "should": [ not cached { "term": { "title": "rabbits" }}, { "term": { "content": "rabbits" }} ]}}, { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ]}} ], "must_not": { "term": { "content": "fox" }} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  83. 83. "bool": { "must": [ { "bool": { "should": [ cached { "term": { "title": "rabbits" }}, { "term": { "content": "rabbits" }} ]}}, { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "content": "quick" }} ]}} ], "must_not": { "term": { "content": "fox" }} } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  84. 84. result bitset = (title:rabbits OR content:rabbits) AND (title:quick OR content:quick) AND NOT content:fox Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  85. 85. filters • boolean yes/no • exact values • cached • faster Filter first, then query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  86. 86. how relevant is this term? Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  87. 87. how relevant is this term? term query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  88. 88. how relevant is this term? term query ≈ term filter + relevance Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  89. 89. how relevant is this term? "term": { "title": "brown" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  90. 90. GET /_search { "query": { } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  91. 91. GET /_search { "query": { "term": { "title": "brown" } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  92. 92. how relevant is this doc? { "_index": "myindex", "_type": "mytype", "_id": "1", "_score": 0.5, "_source": { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  93. 93. relevance score How common is the term in this doc? ➔ more is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  94. 94. relevance score How common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  95. 95. relevance score How common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  96. 96. lucene similarity How common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  97. 97. lucene similarity How common is the term in this doc? ➔ more is better How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  98. 98. lucene similarity Term frequency How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  99. 99. lucene similarity Term frequency How common is the term in ALL docs? ➔ less is better How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  100. 100. lucene similarity Term frequency Inverse document frequency How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  101. 101. lucene similarity Term frequency Inverse document frequency How long is this doc? ➔ shorter is better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  102. 102. lucene similarity Term frequency Inverse document frequency Length norm Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  103. 103. AND … OR … NOT Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  104. 104. AND … OR … NOT bool query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  105. 105. AND … OR … NOT bool query like bool filter, but different... Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  106. 106. "bool": { "must": [ <queries> ], # AND "should": [ <queries> ], "must_not": [ <queries> ] # NOT } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  107. 107. "bool": { "must": [ <queries> ], # AND "should": [ <queries> ], # Hmmm "must_not": [ <queries> ] # NOT } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  108. 108. "bool": { "must": [ <queries> ], # AND "should": [ <queries> ], # Hmmm "must_not": [ <queries> ], # NOT "minimum_should_match": ? # Hmmm } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  109. 109. no "must" queries "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] } at least one must match Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  110. 110. no "must" queries "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] } minimum_should_match = 1 Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  111. 111. with "must" queries "bool": { "must": { "term": { "title": "quick"}}, "should": [ { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] } all are optional! Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  112. 112. with "must" queries "bool": { "must": { "term": { "title": "quick"}}, "should": [ { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] } minimum_should_match = 0 Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  113. 113. bool filter ➔ Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited. T/F
  114. 114. bool filter ➔ Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited. T/F bool query ➔ _score
  115. 115. _score of bool query = Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  116. 116. _score of bool query = sum( _score of each query) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  117. 117. _score of bool query = sum( _score of each query) * num of matching queries Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  118. 118. _score of bool query = sum( _score of each query) * num of matching queries / num of queries Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  119. 119. more matching should queries Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  120. 120. more matching should queries == better relevance score Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  121. 121. trim the long tail "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  122. 122. trim the long tail "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ], "minimum_should_match": "75%" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  123. 123. trim the long tail "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "rabbits" }} ], "minimum_should_match": "75%" # 2 of 3 } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  124. 124. match query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  125. 125. match query high level query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  126. 126. match query high level query understands mapping & analysis Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  127. 127. match query ➔ analyze query string ➔ rewrite query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  128. 128. one word query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  129. 129. { "match": { "title": "QUICK!" }} title:quick { "term": { "title": "quick" }} Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  130. 130. multi word query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  131. 131. { "match": { "title": "QUICK FOX!" }} title:quick OR title:fox { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "fox" }} ] }} Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  132. 132. all words must match Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  133. 133. { "match": { "title": "QUICK FOX!" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  134. 134. { "match": { "title": { "query": "QUICK FOX!", } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  135. 135. { "match": { "title": { "query": "QUICK FOX!", "operator": "and" } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  136. 136. { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "fox" }} ] } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  137. 137. { "bool": { "must": [ { "term": { "title": "quick" }}, { "term": { "title": "fox" }} ] } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  138. 138. trim long tail Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  139. 139. { "match": { "title": { "query": "QUICK BROWN FOX!", "minimum_should_match": "75%" } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  140. 140. { "bool": { "should": [ { "term": { "title": "quick" }}, { "term": { "title": "brown" }}, { "term": { "title": "fox" }} ], "minimum_should_match": 2 } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  141. 141. fuzzy queries levenshtein edit distance Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  142. 142. bron ➔ insertion brown Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  143. 143. bron ➔ deletion brown foxs ➔ fox Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  144. 144. bron substitution ➔ brown foxs ➔ fox kiuck ➔ qiuck Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  145. 145. bron ➔ brown foxs ➔ fox kiuck ➔ qiuck Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited. ➔ quick transposition
  146. 146. { "match": { "title": { "query": "KIUCK BRON FOXS!", "fuzziness": "AUTO" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  147. 147. phrase / proximity Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  148. 148. { "match_phrase": { "title":"QUICK BROWN FOX!" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  149. 149. { "match_phrase": { "title": { "query": "BROWN QUICK FOX!", "slop": "10" } } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  150. 150. combine queries Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  151. 151. "bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  152. 152. "bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  153. 153. "match": { "title": { "query": "<words>", "minimum_should_match": "75%" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  154. 154. "bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  155. 155. "match": { "title": { "query": "<words>", "fuzziness": "AUTO" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  156. 156. "bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  157. 157. "match_phrase": { "title": { "query": "<words>", "slop": "10" } } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  158. 158. "bool": { "must": { <min_should_match> }, "should": [ { <fuzzy> }, { <proximity> } ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  159. 159. multi-field queries Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  160. 160. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  161. 161. easy! Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  162. 162. "bool": { "should": [ { "match": {"first": "Reginald" }}, { "match": {"middle": "Kenneth" }}, { "match": {"last": "Dwight" }} ], "minimum_should_match": "75%" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  163. 163. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  164. 164. hard! Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  165. 165. "bool": { "should": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  166. 166. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  167. 167. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } better match Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  168. 168. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } But 2 matches wins Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  169. 169. dis_max query all docs which match any query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  170. 170. dis_max query all docs which match any query _score = best matching query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  171. 171. "bool": { "should": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  172. 172. "dis_max": { "queries": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  173. 173. { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  174. 174. { "title": "Keeping pets healthy", "content": "My quick brown fox eats rabbits on a regular basis" } { "title": "Quick brown rabbits", "content": "Brown rabbits are commonly seen" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  175. 175. "dis_max": { "queries": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ], "tie_breaker": 0.2 } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  176. 176. dis_max query all docs which match any query _score = best matching query + tie_breaker * others Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  177. 177. multi_match query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  178. 178. multi_match query match query on multiple fields Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  179. 179. "dis_max": { "queries": [ { "match": { "title": "quick brown fox" }}, { "match": { "content": "quick brown fox" }} ], "tie_breaker": 0.2 } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  180. 180. "multi_match": { "query": "quick brown fox", "fields": [ "title", "content" ] "tie_breaker": 0.2 } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  181. 181. "multi_match": { "query": "quick brown fox", "fields": [ "title", "content" ] "tie_breaker": 0.2, # "type": "best_fields" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  182. 182. best_fields find whole "concept" in one field Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  183. 183. best_fields "quick brown fox" in title or content Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  184. 184. best_fields "quick brown fox" in title or content dis_max Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  185. 185. "title": { "type": "string" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  186. 186. "title": { "type": "string", "fields": { }} Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  187. 187. "title": { "type": "string", "fields": { "stemmed": { "type": "string", "analyzer": "english" } }} Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  188. 188. "title": { "type": "string", "fields": { "stemmed": { "type": "string", "analyzer": "english" }, "autocomplete": { "type": "string", "analyzer": "edge_ngrams" }}} Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  189. 189. title: [ brown, fox, jumped ] Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  190. 190. title: [ brown, fox, jumped ] title.stemmed: [ brown, fox, jump ] Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  191. 191. title: [ brown, fox, jumped ] title.stemmed: [ brown, fox, jump ] title.autocomplete [ b, br, bro, brow, brown, f, fo, fox, j, ju, jum, jump, jumpe, jumped ] Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  192. 192. "multi_match": { "query": "quick brown fox", "fields": [ "title", "title.stemmed", "title.autocomplete" ] "type": "most_fields" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  193. 193. most_fields match same text analyzed in different ways Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  194. 194. most_fields more matching fields = better Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  195. 195. most_fields more matching fields = better bool Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  196. 196. { "first": "Reginald", "middle": "Kenneth" "last": "Dwight" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  197. 197. "multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  198. 198. "multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "????" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  199. 199. "multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "most_fields" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  200. 200. problem field centric Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  201. 201. ( first: Reginald OR first: Kenneth OR first: Dwight ) OR ( middle:Reginald OR middle:Kenneth OR middle:Dwight ) OR ( last: Reginald OR last: Kenneth OR last: Dwight ) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  202. 202. ( first: Reginald OR first: Kenneth OR first: Dwight ) OR ( middle:Reginald OR middle:Kenneth OR middle:Dwight ) OR ( last: Reginald OR last: Kenneth OR last: Dwight ) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  203. 203. problem operator: and Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  204. 204. ( first: Reginald AND first: Kenneth AND first: Dwight ) OR ( middle:Reginald AND middle:Kenneth AND middle:Dwight ) OR ( last: Reginald AND last: Kenneth AND last: Dwight ) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  205. 205. ( first: Reginald AND first: Kenneth AND first: Dwight ) OR ( middle:Reginald AND middle:Kenneth AND middle:Dwight ) OR ( last: Reginald AND last: Kenneth AND last: Dwight ) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  206. 206. problem term frequencies first:dwight ➔ common last: dwight ➔ uncommon Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  207. 207. solution Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  208. 208. index time solution single "fullname" field Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  209. 209. "first": { "type": "string" }, "middle": { "type": "string" }, "last": { "type": "string" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  210. 210. "first": { "type": "string" }, "middle": { "type": "string" }, "last": { "type": "string" }, "full": { "type": "string" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  211. 211. "first": { "type": "string", "copy_to": "full" }, "middle": { "type": "string", "copy_to": "full" }, "last": { "type": "string", "copy_to": "full" }, "full": { "type": "string" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  212. 212. "match": { "full": "Reginald Kenneth Dwight", "minimum_should_match": "75%" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  213. 213. query time solution term-centric query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  214. 214. ( first: Reginald AND first: Kenneth AND first: Dwight ) OR ( middle:Reginald AND middle:Kenneth AND middle:Dwight ) OR ( last: Reginald AND last: Kenneth AND last: Dwight ) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  215. 215. ( first: Reginald OR middle:Reginald OR last: Reginald ) AND ( first: Kenneth OR middle:Kenneth OR last: Kenneth ) AND ( first: Dwight OR middle:Dwight OR last: Dwight ) Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  216. 216. blend(first,middle,last):Reginald AND blend(first,middle,last):Kenneth AND blend(first,middle,last):Dwight blends term frequencies Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  217. 217. cross_fields query multiple fields as if they were one Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  218. 218. "multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "cross_fields" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  219. 219. "multi_match": { "query": "Reginald Kenneth Dwight", "fields": [ "first", "middle", "last" ] "type": "cross_fields", "minimum_should_match": "75%" } Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  220. 220. best_fields: whole concept in single field most_fields: same text, different analyzers cross_fields: treat multiple fields as one Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  221. 221. understand the building blocks Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  222. 222. the rest is details Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.
  223. 223. thank you @clintongormley elasticsearch.org/downloads elasticsearch.com/support elasticsearch.com/jobs Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited.

×