More Related Content Similar to Elasticsearch sur Azure : Make sense of your (BIG) data ! (20) Elasticsearch sur Azure : Make sense of your (BIG) data !2. Elasticsearch sur Azure
Make sense of your (BIG) data !
David Pilato / Thomas Conté!
Développeurs - Evangélistes
Elasticsearch / Microsoft
@dadoonet / @tomconte
StartUp
3. Donnez votre avis !
Depuis votre smartphone sur :
http://notes.mstechdays.fr
!
De nombreux lots à gagner toute les heures !!!
Claviers, souris et jeux Microsoft…
!
Merci de nous aider à améliorer les Techdays !
#mstechdays #elasticsearch
StartUp
5. Windows Azure Plateforme Ouverte
Back-ends Android, iOS & Node.js via
Azure Mobile Services
Devices
Languages
SDKs pour Java, Python, PHP, Ruby, Node.js
WebSites pour PHP, Node.js,
Python & App Gallery
Apps
MySQL via ClearDB, MongoDB via MongoLab,
Hadoop via HDInsight
Operating
Systems
Linux VMs via Image Gallery & VMDepot
#mstechdays #elasticsearch
Data
StartUp
6. Windows Azure : rappels
Identity
Cloud
Services
Table
Storage
Blob
Storage
3rd Party
Services
Big Data
Integration
“Plateforme
Ouverte”
Virtual
Machines
Web Sites
Caching
Service
Bus
#mstechdays #elasticsearch
SQL
Azure
Analytics
Mobile
Services
Media
Services
StartUp
CDN
9. search = like % ?
SELECT
doc.*, pays.*
FROM
doc, pays
WHERE
doc.pays_code = pays.code AND
doc.date_doc > to_date('2011-12', 'yyyy-mm') AND
doc.date_doc < to_date('2012-01', 'yyyy-mm') AND
lower(pays.libelle) = 'france' AND
lower(doc.commentaire) LIKE ‘%produit%' AND
lower(doc.commentaire) LIKE ‘%david%';
#mstechdays #elasticsearch
StartUp
19. … and play!
$ curl -XPUT localhost:9200/sessions/session/1 -d '{!
"title"
: "Elasticsearch sur Azure",!
"subtitle" : "Make sense of your (BIG) data !",!
"date"
: "2014-02-13T16:30:00",!
"tags"
: [ "elasticsearch", "azure", "cloud" ],!
"speaker"
: [ {!
"first_name" : "Thomas", !
"last_name" : "Conté" !
}, {!
"first_name" : "David", !
"last_name" : "Pilato" !
}]!
}'
#mstechdays #elasticsearch
StartUp
20. Search!
$ curl -XPOST http://localhost:9200/sessions/session/_search -d'
{
"query": {
"multi_match": {
"query": "elasticsearch azure thomas",
"fields": [ "title^2", "speaker.first_name" ]
}
},
"post_filter": {
"range": {
"date": {
"from": "2014-02-13",
"to": "2014-02-14"
}
}
}
}'
#mstechdays #elasticsearch
StartUp
22. Compute!
$ curl -XPOST http://localhost:9200/sessions/session/_search -d'
{
"query": { ... },
"aggs": {
"by_date": {
"date_histogram": {
"field": "date",
"interval": "day",
"format" : "dd/MM/yyyy"
}
}
}
}'
#mstechdays #elasticsearch
StartUp
23. Compute!
$ curl -XPOST http://localhost:9200/sessions/session/_search -d'
{
"query": { ... },
"aggs": {
"by_date": {
"date_histogram": {
"field": "date",
"interval": "day",
"format" : "dd/MM/yyyy"
}
}
"by_date": [
}
{ "key_as_string": "11/02/2014", "doc_count": 1 },
}'
{ "key_as_string": "12/02/2014", "doc_count": 2 },
{ "key_as_string": "13/02/2014", "doc_count": 3 }
]
#mstechdays #elasticsearch
StartUp
24. Compute!
$ curl -XPOST http://localhost:9200/sessions/session/_search -d'
{
"query": { ... },
"aggs": {
"by_date": {
"date_histogram": {
"field": "date",
"interval": "day",
"format" : "dd/MM/yyyy"
}
}
"by_date": [
}
{ "key_as_string": "11/02/2014", "doc_count": 1 },
}'
{ "key_as_string": "12/02/2014", "doc_count": 2 },
{ "key_as_string": "13/02/2014", "doc_count": 3 }
]
#mstechdays #elasticsearch
StartUp
25. démo
MAKE SENSE OF YOUR (BIG) DATA!
let’s inject some marketing documents…
#mstechdays #elasticsearch
StartUp
28. Distributed indices
node 1
$ curl -XPUT localhost:9200/orders -d '{!
"settings.index.number_of_shards" : 4,!
"settings.index.number_of_replicas" : 1!
}'
#mstechdays #elasticsearch
StartUp
29. Distributed indices
node 1
orders
$ curl -XPUT localhost:9200/orders -d '{!
"settings.index.number_of_shards" : 4,!
"settings.index.number_of_replicas" : 1!
}'
#mstechdays #elasticsearch
StartUp
31. Distributed indices
node 1
orders
1
2
3
4
$ curl -XPUT localhost:9200/orders -d '{!
"settings.index.number_of_shards" : 4,!
"settings.index.number_of_replicas" : 1!
}'
$ curl -XPUT localhost:9200/products -d '{!
"settings.index.number_of_shards" : 2,!
"settings.index.number_of_replicas" : 0!
}'
#mstechdays #elasticsearch
StartUp
32. Distributed indices
node 1
orders
1
2
3
4
products
1
2
#mstechdays #elasticsearch
$ curl -XPUT localhost:9200/orders -d '{!
"settings.index.number_of_shards" : 4,!
"settings.index.number_of_replicas" : 1!
}'
$ curl -XPUT localhost:9200/products -d '{!
"settings.index.number_of_shards" : 2,!
"settings.index.number_of_replicas" : 0!
}'
StartUp
33. Start new nodes
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Armageddon] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Armageddon] detected_master [Ghost Maker]
#mstechdays #elasticsearch
StartUp
34. Start new nodes
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Armageddon] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Armageddon] detected_master [Ghost Maker]
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Cyclops] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Cyclops] detected_master [Ghost Maker]
#mstechdays #elasticsearch
StartUp
35. Start new nodes
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Armageddon] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Armageddon] detected_master [Ghost Maker]
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Cyclops] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Cyclops] detected_master [Ghost Maker]
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Karnak] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Karnak] detected_master [Ghost Maker]
#mstechdays #elasticsearch
StartUp
38. Distributed indices
node 1
node 2
orders
1
2
3
4
products
1
2
#mstechdays #elasticsearch
$ ./elasticsearch-1.0.0/bin/elasticsearch!
[INFO ][node
][Armageddon] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Armageddon] detected_master [Ghost Maker]
StartUp
42. Distributed indices
node 1
node 2
orders
orders
1
2
1
2
3
4
3
4
products
1
#mstechdays #elasticsearch
products
$ ./elasticsearch-1.0.0/bin/elasticsearch!
2
[INFO ][node
][Karnak] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Karnak] detected_master [Ghost Maker]
StartUp
43. Distributed indices
node 1
node 2
orders
node 3
orders
1
2
1
2
3
4
3
4
products
1
#mstechdays #elasticsearch
products
$ ./elasticsearch-1.0.0/bin/elasticsearch!
2
[INFO ][node
][Karnak] {1.0.0}[5645]: initializing
[INFO ][cluster.service][Karnak] detected_master [Ghost Maker]
StartUp
47. Deploying on Azure (unicast)
Cloud service: your-es.cloudapp.net
VM
node 1
private IP1
VM
node 2
private IP2
discovery.zen.ping.multicast.enabled: false!
discovery.zen.ping.unicast.hosts: ["ip1", "ip2"]
#mstechdays #elasticsearch
StartUp
48. Deploying on Azure (unicast)
Cloud service: your-es.cloudapp.net
VM
node 1
private IP1
VM
node 2
private IP2
VM
node 3
private IP3
discovery.zen.ping.multicast.enabled: false!
discovery.zen.ping.unicast.hosts: ["ip1", "ip2", "ip3"]
#mstechdays #elasticsearch
StartUp
49. Deploying on Azure (cloud plugin)
Cloud service: your-es.cloudapp.net
VM
node 1
private IP1
VM
node 2
private IP2
Azure
REST
API
#mstechdays #elasticsearch
StartUp
VM
node 3
private IP3
50. Deploying on Azure (cloud plugin)
Cloud service: your-es.cloudapp.net
VM
node 1
private IP1
Azure
REST
API
VM
node 2
private IP2
cloud:!
azure:!
keystore: /path/to/keystore!
password: your_password_for_keystore!
subscription_id: your_azure_subscription_id!
service_name: your-es!
discovery:!
type: azure
#mstechdays #elasticsearch
StartUp
VM
node 3
private IP3
53. Digital is
business
QUESTIONS ?
@dadoonet / @tomconte
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment
on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
54. Digital is
business
QUESTIONS ?
@dadoonet / @tomconte
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment
on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.