This document introduces Cassandra and Solr for distributed search. It discusses how Cassandra is a distributed data store that powers search at Facebook and how Datastax Enterprise Search uses Cassandra to store data and a Solr index for fast, scalable full-text search. It covers how to set up a Cassandra and Solr cluster for distributed search and write, read, and query data across the cluster in a way that remains available, consistent, and scalable.

1. TOO BIGTO FAIL
An Introduction to Distributed Search with Cassandra and Solr
OpenSource Connections
@PatriciaGorla
pgorla@o19s.com

2. ABOUT ME
Systems Analyst
Programming
Information Retrieval

3. Created at Facebook to
power inbox search
Distributed data store run
on commodity servers
Highly available
No one single point of failure
CASSANDRA

4. WHO USES CASSANDRA?

5. SEARCH + CASSANDRA, 1
• First implementation: Solandra (originally Lucandra)
• Replaced Lucene index with Cassandra column families

6. SEARCH + CASSANDRA, 2
• DataStax Enterprise Search
• Uses native Lucene index
• All data is retrieved from Cassandra

7. Datastax Enterprise Search Cluster
Distributed
Linearly Scalable
Highly Available
Eventually Consistent
Full-text search
Aggregation

8. SETTING UPTHE SCHEMA
<fields>
<field name="id" type="string" indexed="true" stored="true"/>
<field name="name" type="text" indexed="true" stored="true"/>
<field name="body" type="text" indexed="true" stored="true"/>
<field name="title" type="text" indexed="true" stored="true"/>
<field name="date" type="string" indexed="true" stored="true"/>
</fields>

9. WRITINGTO CLUSTER
• Write to either Cassandra clients or Solr API
• Write process is the same
• True atomic updates to Cassandra

10. Cassandra nodes are set up according to row-key hash.

11. Data can be written directly to Cassandra

12. Data is distributed according to row key hash and replication
factor

13. DSE first writes to
Cassandra

14. And then updates the
secondary index on Solr

15. The quorum responds with success / failure

16. Data is now distributed evenly

17. READING FROM CLUSTER
• Read either Cassandra-side or through Solr API
• Cassandra: fast reads*
• Solr: full-text search
• Read direction affects performance
• Data is stored in Cassandra

18. Query is sent to node

19. Node uses gossip to find who has the information

20. QUERYING CASSANDRA
• Can query Solr or Cassandra directly
• Limited syntax with CQL, can use solr_query parameter

21. Querying Cassandra
directly

22. Cassandra retrieves
information from column
family

23. Querying Solr index

24. Row-key hashes are
stored in Solr, and
Cassandra is queried for
stored data

25. Cassandra node sends
request to node with the
corresponding hash,
returns information

26. Data is always synced

27. Both nodes respond with information

28. Updates can be committed and searched over in real time

29. PRODUCTION USE
• Will want a mix of analytics, search nodes

30. An OLTP - OLAP integrated solution

31. TRADEOFFS
• Changing the Solr schema requires reindex (standard for Solr)
• No multi-valued fields or composite columns

32. Q&A
@PatriciaGorla
pgorla@o19s.com
o19s.com/blog