Riak Search 2.0 
Eric Redmond 
@coderoshi
After two years, 
Riak 2.0 is out
┬─┬ ( ^_^ノ) 
Setup 
ノ
riak.conf 
## The enabled Search set this 'on'. 
search = off 
! 
## The port number which Solr binds to. 
search.solr_por...
riak.conf 
## The enabled Search set this 'on'. 
search = on 
! 
## The port number which Solr binds to. 
search.solr_port...
riak-admin cluster join riak@10.0.0.2 
... and so on... 
riak-admin cluster plan 
riak-admin cluster commit
riak-admin cluster join riak@10.0.0.2 
... and so on... 
riak-admin cluster plan 
riak-admin cluster commit 
riak-admin se...
require 'riak' 
client = Riak::Client.new 
! 
# create index 
client.create_search_index('simple') 
! 
# tie the index to ...
Indexing Datatypes
UTF8中搜索 
חיפוש ב UTF8 
Αναζήτηση σε UTF8 
Искать в UTF8 
Search in UTF8
Advanced Search 
aka. Next Level Search
solr = RSolr.connect(url: 'http://yokozuna01.bos1:8098/solr/docs') 
resp = solr.get('select', params: {q: '*:*'})
Facets, Stats, and stuff
eDisMax 
defType=edismax
Highlighting
Languages
Map/Reduce
{"inputs": { 
"module":"yokozuna", 
"function":"mapred_search", 
"arg":["docs","title_s:Key* AND language_s:en"] }, 
"quer...
bucket = Riak::Bucket.new(client, 'people') 
bucket.props = { search_index: 'faces' } 
bucket.get('stuart').store 
bucket ...
riak-admin bucket-type create faces '{"props": 
{"search_index":"faces"}}' 
! 
bucket = Riak::Bucket.new(client, ‘people')...
Schemas
<schema name="default" version="1.5"> 
<fields> 
<field name="kinds" type="string" indexed="true" stored="false" multiValu...
Fallen Nodes
Why?
Thermocline
Thermocline of Replication
Goals
But... why?
How
Yokozuna = Glue
Extractors 
yz_extractor.erl 
yz_json_extractor.erl 
yz_noop_extractor.erl 
yz_text_extractor.erl 
yz_xml_extractor.erl 
y...
Interfaces 
yz_pb_admin.erl 
yz_pb_search.erl 
yz_schema.erl 
yz_wm_extract.erl 
yz_wm_index.erl 
yz_wm_schema.erl 
yz_wm_...
Security
Permissions 
riak-admin security grant search.admin ON schema TO admin 
riak-admin security grant search.admin ON index TO...
yz_pb_admin.erl 
yz_schema.erl 
yz_wm_index.erl 
Administration yz_wm_schema.erl
yz_pb_search.erl 
Searches 
yz_wm_search.erl
yokozuna.erl 
Backend Magic 
yz_app.erl 
yz_general_sup.erl 
yz_kv.erl 
yz_misc.erl 
yz_sup.erl
Distribution 
yz_cover.erl 
yz_events.erl
yz_entropy.erl 
AAE 
yz_entropy_mgr.erl 
yz_exchange_fsm.erl 
yz_index.erl 
yz_index_hashtree.erl 
yz_index_hashtree_sup.e...
Solr code 
EntropyData.java 
Monitor.java 
yz_solr.erl 
yz_solr_proc.erl 
yz_solr_sup.erl
Stats 
yz_stat.erl 
yz_stat_worker.erl
“Testing only shows 
the presence, not 
the absence, 
of bugs” 
aae_test.erl 
yokozuna_essential.erl 
yz_errors.erl 
yz_fa...
Thanks 
Eric Redmond 
@coderoshi
Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014
Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014
Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014
Nächste SlideShare
Wird geladen in …5
×

Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014

684 Aufrufe

Veröffentlicht am

Eric Redmond – Distributed Search on Riak 2.0

Riak excels at one type of query: key puts and key gets. But the world demands more from a database. Since Basho isn't primarily a search company, we decided to leverage the power of Solr for Riak 2.0. This is a walkthrough of what new features we added, how works, and why you'd want to use it. Also, of course, live demos.

Veröffentlicht in: Daten & Analysen
0 Kommentare
1 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
684
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
3
Aktionen
Geteilt
0
Downloads
9
Kommentare
0
Gefällt mir
1
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014

  1. 1. Riak Search 2.0 Eric Redmond @coderoshi
  2. 2. After two years, Riak 2.0 is out
  3. 3. ┬─┬ ( ^_^ノ) Setup ノ
  4. 4. riak.conf ## The enabled Search set this 'on'. search = off ! ## The port number which Solr binds to. search.solr_port = 10014 ! ## The port number which Solr JMX binds to. search.solr_jmx_port = 10013 ! ## The arguments to pass to the Solr JVM. Non-standard ## arguments, i.e. -XX, may not be portable across JVM ## implementations. E.g. -XX:+UseCompressedStrings. search.solr_jvm_args = -Xms1g -Xmx1g -XX:+UseStringCache - XX:+UseCompressedOops
  5. 5. riak.conf ## The enabled Search set this 'on'. search = on ! ## The port number which Solr binds to. search.solr_port = 10014 ! ## The port number which Solr JMX binds to. search.solr_jmx_port = 10013 ! ## The arguments to pass to the Solr JVM. Non-standard ## arguments, i.e. -XX, may not be portable across JVM ## implementations. E.g. -XX:+UseCompressedStrings. search.solr_jvm_args = -Xms1g -Xmx1g -XX:+UseStringCache - XX:+UseCompressedOops
  6. 6. riak-admin cluster join riak@10.0.0.2 ... and so on... riak-admin cluster plan riak-admin cluster commit
  7. 7. riak-admin cluster join riak@10.0.0.2 ... and so on... riak-admin cluster plan riak-admin cluster commit riak-admin security add-user eric 12345 riak-admin security add-user admin 123456 riak-admin security grant search.query ON index simple TO any riak-admin security grant search.admin ON schema TO admin
  8. 8. require 'riak' client = Riak::Client.new ! # create index client.create_search_index('simple') ! # tie the index to bucket 'cats' bucket = Riak::Bucket.new(client, 'cats') bucket.props = { search_index: 'simple' }
  9. 9. Indexing Datatypes
  10. 10. UTF8中搜索 חיפוש ב UTF8 Αναζήτηση σε UTF8 Искать в UTF8 Search in UTF8
  11. 11. Advanced Search aka. Next Level Search
  12. 12. solr = RSolr.connect(url: 'http://yokozuna01.bos1:8098/solr/docs') resp = solr.get('select', params: {q: '*:*'})
  13. 13. Facets, Stats, and stuff
  14. 14. eDisMax defType=edismax
  15. 15. Highlighting
  16. 16. Languages
  17. 17. Map/Reduce
  18. 18. {"inputs": { "module":"yokozuna", "function":"mapred_search", "arg":["docs","title_s:Key* AND language_s:en"] }, "query":[ {"map":{ "language":"javascript", "keep":false, "source":"function(v) { return [1]; }"}}, {"reduce"{ "language":"javascript", "keep":true, "name":"Riak.reduceSum" }} ] }
  19. 19. bucket = Riak::Bucket.new(client, 'people') bucket.props = { search_index: 'faces' } bucket.get('stuart').store bucket = Riak::Bucket.new(client, 'cats') bucket.props = { search_index: 'faces' } bucket.get('stuart').store bucket = Riak::Bucket.new(client, 'mountains') bucket.props = { search_index: 'faces' } bucket.get('hood').store
  20. 20. riak-admin bucket-type create faces '{"props": {"search_index":"faces"}}' ! bucket = Riak::Bucket.new(client, ‘people') bucket.get('stuart', type:'faces').store bucket = Riak::Bucket.new(client, ‘cats') bucket.get('stuart', type:'faces').store bucket = Riak::Bucket.new(client, ‘mountains') bucket.get('stuart', type:'faces').store
  21. 21. Schemas
  22. 22. <schema name="default" version="1.5"> <fields> <field name="kinds" type="string" indexed="true" stored="false" multiValued="true" /> <field name="name" type="string" indexed="true" stored="true" /> ! <dynamicField name="*_ss" type="string" indexed="true" stored="true" multiValued="true"/> ! <field name="_yz_ed" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_pn" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_fpn" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_vtag" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_node" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_rk" type="_yz_str" indexed="true" stored="true"/> <field name="_yz_rb" type="_yz_str" indexed="true" stored="true"/> </fields> </schema>
  23. 23. Fallen Nodes
  24. 24. Why?
  25. 25. Thermocline
  26. 26. Thermocline of Replication
  27. 27. Goals
  28. 28. But... why?
  29. 29. How
  30. 30. Yokozuna = Glue
  31. 31. Extractors yz_extractor.erl yz_json_extractor.erl yz_noop_extractor.erl yz_text_extractor.erl yz_xml_extractor.erl yz_doc.erl
  32. 32. Interfaces yz_pb_admin.erl yz_pb_search.erl yz_schema.erl yz_wm_extract.erl yz_wm_index.erl yz_wm_schema.erl yz_wm_search.erl
  33. 33. Security
  34. 34. Permissions riak-admin security grant search.admin ON schema TO admin riak-admin security grant search.admin ON index TO admin riak-admin security grant search.query ON index TO user riak-admin security grant search.query ON index wiki TO user
  35. 35. yz_pb_admin.erl yz_schema.erl yz_wm_index.erl Administration yz_wm_schema.erl
  36. 36. yz_pb_search.erl Searches yz_wm_search.erl
  37. 37. yokozuna.erl Backend Magic yz_app.erl yz_general_sup.erl yz_kv.erl yz_misc.erl yz_sup.erl
  38. 38. Distribution yz_cover.erl yz_events.erl
  39. 39. yz_entropy.erl AAE yz_entropy_mgr.erl yz_exchange_fsm.erl yz_index.erl yz_index_hashtree.erl yz_index_hashtree_sup.erl
  40. 40. Solr code EntropyData.java Monitor.java yz_solr.erl yz_solr_proc.erl yz_solr_sup.erl
  41. 41. Stats yz_stat.erl yz_stat_worker.erl
  42. 42. “Testing only shows the presence, not the absence, of bugs” aae_test.erl yokozuna_essential.erl yz_errors.erl yz_fallback.erl yz_flag_transitions.erl yz_index_admin.erl yz_languages.erl yz_mapreduce.erl yz_monitor_solr.erl yz_pb.erl yz_rs_migration.erl yz_rt.erl yz_schema_admin.erl yz_siblings.erl yz_solr_start_timeout.erl yz_stat_test.erl yz_wm_extract_test.erl yz_component_tests.erl yz_json_extractor_tests.erl yz_kv_tests.erl yz_misc_tests.erl yz_text_extractor_tests.erl yz_xml_extractor_tests.erl yz_driver.erl yz_file_terms.erl Dijkstra
  43. 43. Thanks Eric Redmond @coderoshi

×