Downloaden Sie, um offline zu lesen
![Indizierung
curl -XPOST "http://localhost:9200/library/book" -d'
{
"title": "Elasticsearch in Action",
"author": [ "Radu Gheorghe",
"Matthew Lee Hinman",
"Roy Russo" ],
"published": "2015-06-30T00:00:00.000Z",
"publisher": {
"name": "Manning",
"country": "USA"
}
}'](https://image.slidesharecdn.com/einfuehrung-in-elasticsearch-151009033527-lva1-app6892/85/Einfuhrung-in-Elasticsearch-6-320.jpg)
![Suche
curl -XPOST "http://localhost:9200/library/book/_search
?q=elasticsearch"
{
[...]
"hits": {
"hits": [
{
"_index": "library",
"_type": "book",
"_source": {
"title": "Elasticsearch in Action",
[...]](https://image.slidesharecdn.com/einfuehrung-in-elasticsearch-151009033527-lva1-app6892/85/Einfuhrung-in-Elasticsearch-7-320.jpg)
![Sprachspezifische Inhalte
curl -XPOST "http://localhost:9200/library/book/" -d'
{
"title": "Elasticsearch - Ein praktischer Einstieg",
"author": "Florian Hopf",
"published": "2015-10-26T00:00:00.000Z",
"publisher": {
"name": "dpunkt.verlag",
"country": "DE"
},
"tags": ["Lucene", "Elasticsearch"]
}'](https://image.slidesharecdn.com/einfuehrung-in-elasticsearch-151009033527-lva1-app6892/85/Einfuhrung-in-Elasticsearch-9-320.jpg)
![Terms-Aggregation
"aggregations": {
"common-tags": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "Elasticsearch",
"doc_count": 2
},
{
"key": "Lucene",
"doc_count": 2
},
{
"key": "Java",
"doc_count": 1
}]
[...]](https://image.slidesharecdn.com/einfuehrung-in-elasticsearch-151009033527-lva1-app6892/85/Einfuhrung-in-Elasticsearch-47-320.jpg)
![Aggregationen kombinieren
"buckets": [
{
"key": "Elasticsearch",
"doc_count": 2,
"published": {
"buckets": [
{
"key_as_string": "2015-10-
01T00:00:00.000Z",
"key": 1443657600000,
"doc_count": 2
}
]
}
},
[...]](https://image.slidesharecdn.com/einfuehrung-in-elasticsearch-151009033527-lva1-app6892/85/Einfuhrung-in-Elasticsearch-50-320.jpg)
Einführung in Elasticsearch
- 1.
- 2.
- 3.
- 4. Installation # download archive wgethttps://download.elastic.co/elasticsearch /elasticsearch/elasticsearch-1.7.2.zip # zip is for windows and linux unzip elasticsearch-1.7.2.zip # on windows: elasticsearch.bat elasticsearch-1.7.2/bin/elasticsearch
- 5. Zugriff per HTTP curl-XGET "http://localhost:9200" { "status": 200, "name": "Ultron", "cluster_name": "elasticsearch", "version": { "number" : "1.7.2", "build_hash" : "e43676b1385b8125...", "build_timestamp" : "2015-09-14T09:49:53Z", "build_snapshot" : false, "lucene_version" : "4.10.4" }, "tagline": "You Know, for Search" }
- 6. Indizierung curl -XPOST "http://localhost:9200/library/book"-d' { "title": "Elasticsearch in Action", "author": [ "Radu Gheorghe", "Matthew Lee Hinman", "Roy Russo" ], "published": "2015-06-30T00:00:00.000Z", "publisher": { "name": "Manning", "country": "USA" } }'
- 7. Suche curl -XPOST "http://localhost:9200/library/book/_search ?q=elasticsearch" { [...] "hits":{ "hits": [ { "_index": "library", "_type": "book", "_source": { "title": "Elasticsearch in Action", [...]
- 8. Suche per QueryDSL curl -XPOST "http://localhost:9200/library/book/_search" -d' { "query": { "filtered": { "query": { "match": { "title": "elasticsearch" } }, "filter": { "term": { "publisher.name": "manning" } } } } }'
- 9. Sprachspezifische Inhalte curl -XPOST"http://localhost:9200/library/book/" -d' { "title": "Elasticsearch - Ein praktischer Einstieg", "author": "Florian Hopf", "published": "2015-10-26T00:00:00.000Z", "publisher": { "name": "dpunkt.verlag", "country": "DE" }, "tags": ["Lucene", "Elasticsearch"] }'
- 10. Sprachspezifische Inhalte curl -XPOST"http://localhost:9200/library/book/_search" -d' { "query": { "filtered": { "query": { "match": { "title": "praktisch" } } } } }'
- 11. Analyzing Term Document Id Action1 ein 2 Einstieg 2 Elasticsearch 1,2 in 1 praktischer 2 1. Tokenization Elasticsearch in Action Elasticsearch: Ein praktischer Einstieg
- 12. Analyzing Term Document Id action1 ein 2 einstieg 2 elasticsearch 1,2 in 1 praktischer 2 1. Tokenization Elasticsearch in Action Elasticsearch: Ein praktischer Einstieg 2. Lowercasing
- 13. Suche Term Document Id action1 ein 2 einstieg 2 elasticsearch 1,2 in 1 praktischer 2 1. Tokenization 2. LowercasingElasticsearch elasticsearch
- 14. Suche Term Document Id action1 ein 2 einstieg 2 elasticsearch 1,2 in 1 praktischer 2 1. Tokenization 2. Lowercasingpraktisch praktisch
- 15. Analyzing Term Document Id action1 ein 2 einstieg 2 elasticsearch 1,2 in 1 praktisch 2 1. Tokenization Elasticsearch in Action Elasticsearch: Ein praktischer Einstieg 2. Lowercasing 3. Stemming
- 16. Suche Term Document Id action1 ein 2 einstieg 2 elasticsearch 1,2 in 1 praktisch 2 1. Tokenization 2. Lowercasingpraktisch praktisch
- 17. Mapping curl -XPUT "http://localhost:9200/library/book/_mapping" -d' { "book":{ "properties": { "title": { "type": "string", "analyzer": "german" } } } }'
- 18. Suchfeatures ● Fertige Analyzer, Konfigurationvon eigenen ● Relevanzberechnung ● Paginierung, Sortierung ● Highlighting, Autovervollständigung, ... ● Facettierung über Aggregationen
- 19. Recap ● Java-basierter Suchserver ● Kommunikation überHTTP und JSON ● Dokumentenbasierte Speicherung ● Unterstützung unterschiedlicher Datentypen ● Suche über Query DSL auf invertiertem Index
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26. Recap ● Knoten können Clusterbilden ● Datenverteilung durch Sharding ● Replicas zur Lastverteilung und Ausfallsicherheit ● Verteilte Suche ● Cluster-Zustand auf jedem Knoten
- 27.
- 28. Java! dependencies { compile group:'org.elasticsearch', name: 'elasticsearch', version: '1.7.2' }
- 29. Java! TransportAddress address = newInetSocketTransportAddress("localhost", 9300); Client client = new TransportClient(). addTransportAddress(address);
- 30. Suche per QueryDSL curl -XPOST "http://localhost:9200/library/book/_search" -d' { "query": { "filtered": { "query": { "match": { "title": "elasticsearch" } }, "filter": { "term": { "publisher.name": "manning" } } } } }'
- 31. Suche über Java-Client SearchResponsesearchResponse = client .prepareSearch("library") .setQuery(filteredQuery( matchQuery("title", "elasticsearch"), termFilter("publisher.name", "manning"))) .execute().actionGet(); assertEquals(1, searchResponse.getHits().getTotalHits()); SearchHit hit = searchResponse.getHits().getAt(0); assertEquals("Elasticsearch in Action", hit.getSource().get("title"));
- 32. Indizierung über Java-Client XContentBuilderjsonBuilder = jsonBuilder() .startObject() .field("title", "Elasticsearch") .field("author", "Florian Hopf") .startObject("publisher") .field("name", "dpunkt") .endObject() .endObject(); client.prepareIndex("library", "book") .setSource(jsonBuilder) .execute().actionGet();
- 33.
- 34. Transport-Client ● Verbindet sich mitbestehendem Cluster TransportAddress address = new InetSocketTransportAddress("localhost", 9300); Client client = new TransportClient(). addTransportAddress(address);
- 35.
- 36. Node-Client Node node =nodeBuilder().client(true).node(); Client client = node.client();
- 37. Node-Client ● Startet eigenen Knoten ● Anwendungwird Teil des Clusters Node node = nodeBuilder().client(true).node(); Client client = node.client();
- 38.
- 39.
- 40. Standard-Client Gotchas ● Elasticsearch-Version Client== Server ● Node-Client ● Bei Start und Stop wird Cluster-State übertragen ● Speicherverbrauch ● Elasticsearch-Dependency
- 41. Alternativen ● Jest: Http-Client ● Spring DataElasticsearch ● Große Auswahl für JVM-Sprachen
- 42.
- 43. Aggregations ● Informationen über dieDaten ● Ersetzen und ermöglichen Facetten ● Anwendungsfälle für Suchanwendungen und Analytics
- 44.
- 45. Invertierter Index Term DocumentId Elasticsearch 1,2,3 Java 3 Lucene 2,3
- 46. Terms-Aggregation curl -XPOST "http://localhost:9200/library/book/_search" -d' { "size":0, "aggs": { "common-tags": { "terms": { "field": "tags" } } } }'
- 47. Terms-Aggregation "aggregations": { "common-tags": { "doc_count_error_upper_bound":0, "sum_other_doc_count": 0, "buckets": [ { "key": "Elasticsearch", "doc_count": 2 }, { "key": "Lucene", "doc_count": 2 }, { "key": "Java", "doc_count": 1 }] [...]
- 48.
- 49. Aggregationen kombinieren GET /library/book/_search { "aggs":{ "tags": { "terms": { "field": "tags.raw", "size": 10 }, "aggs": { "tag_published": { "date_histogram": { "field": "published", "interval": "month" } } } } } }
- 50. Aggregationen kombinieren "buckets": [ { "key":"Elasticsearch", "doc_count": 2, "published": { "buckets": [ { "key_as_string": "2015-10- 01T00:00:00.000Z", "key": 1443657600000, "doc_count": 2 } ] } }, [...]
- 51. Metric-Aggregationen ● Berechnen einen odermehrere Werte ● Meist auf numerischen Feldern ● Stats, Percentiles, Min, Max, Sum, Avg, ...
- 52. Stats-Aggregationen GET /library/book/_search { "aggs": { "published_stats":{ "stats": { "field": "published" } } } }
- 53. Stats-Aggregationen "aggregations": { "published_stats": { "count":5, "min": 1419292800000, "max": 1445990400000, "avg": 1440547200000, "sum": 7202736000000, "min_as_string": "2014-12-23T00:00:00.000Z", "max_as_string": "2015-10-28T00:00:00.000Z", "avg_as_string": "2015-08-26T00:00:00.000Z", "sum_as_string": "2198-03-31T00:00:00.000Z" } }
- 54. Recap ● Aggregationen bieten unterschiedliche Einblickein die Daten ● Facettierung ● Kombination mehrerer Aggregationen ● Grundlage für Visualisierungen
- 55.
- 56. Logfile-Analyse ● Zentralisierung Logs ausAnwendungen ● Zentralisierung Logs über Maschinen ● Auch ohne Zugriff ● Leichte Durchsuchbarkeit ● Real-Time-Analysis / Visualisierung ● Daten für alle!
- 57.
- 58.
- 59. Logstash-Config input { file { path=> "/var/log/apache2/access.log" } } filter { grok { match => { message => "%{COMBINEDAPACHELOG}" } } } output { elasticsearch_http { host => "localhost" } }
- 60.
- 61. Recap ● Einlesen, Anreichern, Speichernvon Logevents ● Zahlreiche Inputs in Logstash ● Konsolidierung ● Zentralisierung ● Auswertung
- 62.
- 63. Weitere Infos ● http://elastic.co ● Elasticsearch –The definitive Guide ● https://www.elastic.co/guide/en/elasticsearch/gui de/master/index.html ● Elasticsearch in Action ● https://www.manning.com/books/elasticsearch-in- action ● http://blog.florian-hopf.de