Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Riak Search 2.0 
Eric Redmond 
@coderoshi
After two years, 
Riak 2.0 is out
┬─┬ ( ^_^ノ) 
Setup 
ノ
riak.conf 
## The enabled Search set this 'on'. 
search = off 
! 
## The port number which Solr binds to. 
search.solr_por...
riak.conf 
## The enabled Search set this 'on'. 
search = on 
! 
## The port number which Solr binds to. 
search.solr_port...
riak-admin cluster join riak@10.0.0.2 
... and so on... 
riak-admin cluster plan 
riak-admin cluster commit
riak-admin cluster join riak@10.0.0.2 
... and so on... 
riak-admin cluster plan 
riak-admin cluster commit 
riak-admin se...
require 'riak' 
client = Riak::Client.new 
! 
# create index 
client.create_search_index('simple') 
! 
# tie the index to ...
Indexing Datatypes
UTF8中搜索 
חיפוש ב UTF8 
Αναζήτηση σε UTF8 
Искать в UTF8 
Search in UTF8
Advanced Search 
aka. Next Level Search
solr = RSolr.connect(url: 'http://yokozuna01.bos1:8098/solr/docs') 
resp = solr.get('select', params: {q: '*:*'})
Facets, Stats, and stuff
eDisMax 
defType=edismax
Highlighting
Languages
Map/Reduce
{"inputs": { 
"module":"yokozuna", 
"function":"mapred_search", 
"arg":["docs","title_s:Key* AND language_s:en"] }, 
"quer...
bucket = Riak::Bucket.new(client, 'people') 
bucket.props = { search_index: 'faces' } 
bucket.get('stuart').store 
bucket ...
riak-admin bucket-type create faces '{"props": 
{"search_index":"faces"}}' 
! 
bucket = Riak::Bucket.new(client, ‘people')...
Schemas
<schema name="default" version="1.5"> 
<fields> 
<field name="kinds" type="string" indexed="true" stored="false" multiValu...
Fallen Nodes
Why?
Thermocline
Thermocline of Replication
Goals
But... why?
How
Yokozuna = Glue
Extractors 
yz_extractor.erl 
yz_json_extractor.erl 
yz_noop_extractor.erl 
yz_text_extractor.erl 
yz_xml_extractor.erl 
y...
Interfaces 
yz_pb_admin.erl 
yz_pb_search.erl 
yz_schema.erl 
yz_wm_extract.erl 
yz_wm_index.erl 
yz_wm_schema.erl 
yz_wm_...
Security
Permissions 
riak-admin security grant search.admin ON schema TO admin 
riak-admin security grant search.admin ON index TO...
yz_pb_admin.erl 
yz_schema.erl 
yz_wm_index.erl 
Administration yz_wm_schema.erl
yz_pb_search.erl 
Searches 
yz_wm_search.erl
yokozuna.erl 
Backend Magic 
yz_app.erl 
yz_general_sup.erl 
yz_kv.erl 
yz_misc.erl 
yz_sup.erl
Distribution 
yz_cover.erl 
yz_events.erl
yz_entropy.erl 
AAE 
yz_entropy_mgr.erl 
yz_exchange_fsm.erl 
yz_index.erl 
yz_index_hashtree.erl 
yz_index_hashtree_sup.e...
Solr code 
EntropyData.java 
Monitor.java 
yz_solr.erl 
yz_solr_proc.erl 
yz_solr_sup.erl
Stats 
yz_stat.erl 
yz_stat_worker.erl
“Testing only shows 
the presence, not 
the absence, 
of bugs” 
aae_test.erl 
yokozuna_essential.erl 
yz_errors.erl 
yz_fa...
Thanks 
Eric Redmond 
@coderoshi
Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014
Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014
Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014
Nächste SlideShare
Wird geladen in …5
×

Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014

771 Aufrufe

Veröffentlicht am

Eric Redmond – Distributed Search on Riak 2.0

Riak excels at one type of query: key puts and key gets. But the world demands more from a database. Since Basho isn't primarily a search company, we decided to leverage the power of Solr for Riak 2.0. This is a walkthrough of what new features we added, how works, and why you'd want to use it. Also, of course, live demos.

Veröffentlicht in: Daten & Analysen
  • Als Erste(r) kommentieren

Eric Redmond – Distributed Search on Riak 2.0 - NoSQL matters Barcelona 2014

  1. 1. Riak Search 2.0 Eric Redmond @coderoshi
  2. 2. After two years, Riak 2.0 is out
  3. 3. ┬─┬ ( ^_^ノ) Setup ノ
  4. 4. riak.conf ## The enabled Search set this 'on'. search = off ! ## The port number which Solr binds to. search.solr_port = 10014 ! ## The port number which Solr JMX binds to. search.solr_jmx_port = 10013 ! ## The arguments to pass to the Solr JVM. Non-standard ## arguments, i.e. -XX, may not be portable across JVM ## implementations. E.g. -XX:+UseCompressedStrings. search.solr_jvm_args = -Xms1g -Xmx1g -XX:+UseStringCache - XX:+UseCompressedOops
  5. 5. riak.conf ## The enabled Search set this 'on'. search = on ! ## The port number which Solr binds to. search.solr_port = 10014 ! ## The port number which Solr JMX binds to. search.solr_jmx_port = 10013 ! ## The arguments to pass to the Solr JVM. Non-standard ## arguments, i.e. -XX, may not be portable across JVM ## implementations. E.g. -XX:+UseCompressedStrings. search.solr_jvm_args = -Xms1g -Xmx1g -XX:+UseStringCache - XX:+UseCompressedOops
  6. 6. riak-admin cluster join riak@10.0.0.2 ... and so on... riak-admin cluster plan riak-admin cluster commit
  7. 7. riak-admin cluster join riak@10.0.0.2 ... and so on... riak-admin cluster plan riak-admin cluster commit riak-admin security add-user eric 12345 riak-admin security add-user admin 123456 riak-admin security grant search.query ON index simple TO any riak-admin security grant search.admin ON schema TO admin
  8. 8. require 'riak' client = Riak::Client.new ! # create index client.create_search_index('simple') ! # tie the index to bucket 'cats' bucket = Riak::Bucket.new(client, 'cats') bucket.props = { search_index: 'simple' }
  9. 9. Indexing Datatypes
  10. 10. UTF8中搜索 חיפוש ב UTF8 Αναζήτηση σε UTF8 Искать в UTF8 Search in UTF8
  11. 11. Advanced Search aka. Next Level Search
  12. 12. solr = RSolr.connect(url: 'http://yokozuna01.bos1:8098/solr/docs') resp = solr.get('select', params: {q: '*:*'})
  13. 13. Facets, Stats, and stuff
  14. 14. eDisMax defType=edismax
  15. 15. Highlighting
  16. 16. Languages
  17. 17. Map/Reduce
  18. 18. {"inputs": { "module":"yokozuna", "function":"mapred_search", "arg":["docs","title_s:Key* AND language_s:en"] }, "query":[ {"map":{ "language":"javascript", "keep":false, "source":"function(v) { return [1]; }"}}, {"reduce"{ "language":"javascript", "keep":true, "name":"Riak.reduceSum" }} ] }
  19. 19. bucket = Riak::Bucket.new(client, 'people') bucket.props = { search_index: 'faces' } bucket.get('stuart').store bucket = Riak::Bucket.new(client, 'cats') bucket.props = { search_index: 'faces' } bucket.get('stuart').store bucket = Riak::Bucket.new(client, 'mountains') bucket.props = { search_index: 'faces' } bucket.get('hood').store
  20. 20. riak-admin bucket-type create faces '{"props": {"search_index":"faces"}}' ! bucket = Riak::Bucket.new(client, ‘people') bucket.get('stuart', type:'faces').store bucket = Riak::Bucket.new(client, ‘cats') bucket.get('stuart', type:'faces').store bucket = Riak::Bucket.new(client, ‘mountains') bucket.get('stuart', type:'faces').store
  21. 21. Schemas
  22. 22. <schema name="default" version="1.5"> <fields> <field name="kinds" type="string" indexed="true" stored="false" multiValued="true" /> <field name="name" type="string" indexed="true" stored="true" /> ! <dynamicField name="*_ss" type="string" indexed="true" stored="true" multiValued="true"/> ! <field name="_yz_ed" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_pn" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_fpn" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_vtag" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_node" type="_yz_str" indexed="true" stored="false"/> <field name="_yz_rk" type="_yz_str" indexed="true" stored="true"/> <field name="_yz_rb" type="_yz_str" indexed="true" stored="true"/> </fields> </schema>
  23. 23. Fallen Nodes
  24. 24. Why?
  25. 25. Thermocline
  26. 26. Thermocline of Replication
  27. 27. Goals
  28. 28. But... why?
  29. 29. How
  30. 30. Yokozuna = Glue
  31. 31. Extractors yz_extractor.erl yz_json_extractor.erl yz_noop_extractor.erl yz_text_extractor.erl yz_xml_extractor.erl yz_doc.erl
  32. 32. Interfaces yz_pb_admin.erl yz_pb_search.erl yz_schema.erl yz_wm_extract.erl yz_wm_index.erl yz_wm_schema.erl yz_wm_search.erl
  33. 33. Security
  34. 34. Permissions riak-admin security grant search.admin ON schema TO admin riak-admin security grant search.admin ON index TO admin riak-admin security grant search.query ON index TO user riak-admin security grant search.query ON index wiki TO user
  35. 35. yz_pb_admin.erl yz_schema.erl yz_wm_index.erl Administration yz_wm_schema.erl
  36. 36. yz_pb_search.erl Searches yz_wm_search.erl
  37. 37. yokozuna.erl Backend Magic yz_app.erl yz_general_sup.erl yz_kv.erl yz_misc.erl yz_sup.erl
  38. 38. Distribution yz_cover.erl yz_events.erl
  39. 39. yz_entropy.erl AAE yz_entropy_mgr.erl yz_exchange_fsm.erl yz_index.erl yz_index_hashtree.erl yz_index_hashtree_sup.erl
  40. 40. Solr code EntropyData.java Monitor.java yz_solr.erl yz_solr_proc.erl yz_solr_sup.erl
  41. 41. Stats yz_stat.erl yz_stat_worker.erl
  42. 42. “Testing only shows the presence, not the absence, of bugs” aae_test.erl yokozuna_essential.erl yz_errors.erl yz_fallback.erl yz_flag_transitions.erl yz_index_admin.erl yz_languages.erl yz_mapreduce.erl yz_monitor_solr.erl yz_pb.erl yz_rs_migration.erl yz_rt.erl yz_schema_admin.erl yz_siblings.erl yz_solr_start_timeout.erl yz_stat_test.erl yz_wm_extract_test.erl yz_component_tests.erl yz_json_extractor_tests.erl yz_kv_tests.erl yz_misc_tests.erl yz_text_extractor_tests.erl yz_xml_extractor_tests.erl yz_driver.erl yz_file_terms.erl Dijkstra
  43. 43. Thanks Eric Redmond @coderoshi

×