An introduction to Solr. Presented at JavaZone 2012.

1. SOLR 101
JavaZone 2012, Oslo, Sébastien Muller, Findwise

2. Agenda
 Introductions
 Enterprise search
 What is Solr, why choose it?
 Solr Terminology
 Main Solr Features
 How it works
 Anatomy of a Query
 Scalbility
 Case studies
 Sparebank1
 Komplett Group

3. Enterprise Search
 Search has become mission critical for most enterprises
 Intranet
 Web presence
 E-commerce
 Exponential growth of data
 Cost of not finding information
 Knowledge (sharing)
 Time
 Money
 Information blackhole

4. What is Solr?
Official definition:
“Solr is an open source enterprise search platform
based on the
Lucene Java search library, with an HTTP
interface using XML,
JSON or other formats. It provides hit
highlighting, faceted
search, caching, replication, a web
administration interface and
many more features. It runs in a
Java servlet container such as
Apache Tomcat.”
http://lucene.apache.org/solr

5. What is Solr?
 Open-source, license-free search engine
 Uses Apache Lucene library and adds enterprise search server
features and capabilities
 Web based application that processes requests and returns
responses via HTTP
 Easy scalability and great performance
 Industry-tested worldwide
 Modern solution architecture based on XML and Java – easy to work
with
 Well integrated with the ecosystem around Big Data, such as
Hadoop (also Nutch, Tika).

6. Why choose Solr?
 “Buy” > Build
 Open source vs. Commercial solution
 Open source software is free
 Licensed software can be very expensive
 High quality and easily modifiable relevancy
 Very fast query and indexing performance
 Highly flexible data processing/transformation

7. Why choose Solr?
 Some challenges unique to open source…
 No guaranteed support or bug fixing from community
 No formal quality control or support for upgrades
 Limited support for less experienced developers
 Some benefits unique to open source…
 Widely used and tested
 Access to source code
 Access to development versions and unreleased patches
 Ultimately search is a specialised field and requires specialists

8. Solr Terminology
 Index(ing)
 Inverted index
 Document
 Field
 Stored and/or indexed fields
 Analysis
 Tokenization
 Filters
 Terms
 Query
 Filter
 Function
 Facet

9. Main Solr Features
 Full text search
 Field search
 Number and date searching
 Facets
 Spelling assistance – “Did you mean…?”
 Replication
 Master/Slave architecture
 Related hits
 Query completion
 Admin GUI

10. How it works
 Easy configuration through XML
 schema.xml
 solrconfig.xml
 Documents are POSTed via HTTP to Solr
 Add/update
 Delete
 Commit
 Queries and response are also sent via HTTP
 Choice of formats

11. Anatomy of a Query
 Common parameters
 Start, rows, fl, fq, sort
 http://wiki.apache.org/solr/CommonQueryParameters
?
q=*:*&start=0&rows=10&fl=title&fq=collection:popular&s
ort=title asc
 Slightly more advanced
 &facets
 &qf

12. What is Facetting?
 Navigation/discovery technique
 Tally of docs for each distinct field value
 Parameters
 &facet=true
 &facet.field=category
And so much more…

13. Scalability
 Architecture goals:
 More queries per second (qps)
 Faster query execution
 Bigger indexes
 Faster indexing
 Scaling options
 Multicore
 Replication
 Sharding

14. Scalability - Multicore
 Having more than one Solr in one Solr webapp
<solr persistent = “true” sharedLib = “lib” >
<cores adminPath=“/admin/cores”>
<core name=“core0” instanceDir=“core0” />
<core name=“core1” instanceDir=“core1” />
</cores
</solr>
 http://localhost:8080/solr/admin/cores?action=...
 STATUS
 CREATE
 SWAP

15. Scalability - Replication
 Basic architecture – indexing/querying handled by one instance
 1:1 Master/slave
 Indexing
 Querying
 1:N Master/slaves
 Different user groups

16. Scalability - Sharding
 Distributed index
 N masters with index split between them
 Simple hashing to choose index
 Sharding + replication
 N masters with M slaves each
 More shards = faster execution time
 More slaves = higher average QPS
&shards=solr1:8983/solr,solr2:8983/
solr&indent=true&q=ipod+solr

17. Case Studies

18. SpareBank1 - Background
 SpareBank1 Gruppen
 19 individual localised bank portals and one parent front page
 Boost 25 umbrella project
 Semantic URLs: https://www2.sparebank1.no/9898/3_privat?
_nfpb=true&_nfls=false&_pageLabel=page_privat_innhold&pId=1233
149354625&_
 New search interface
 Banking app
 CMS with no easy way of tracking individual banks’
publications
 Mass duplicates
 Access to irrelevant data

19. SpareBank1 - Requirements
 Customer requirements : “bedre portal søk”

20. SpareBank1 - Requirements
 Basic search features include
 High quality relevance and precision
 Relevant faceting
 Query completion
 Spell check and suggestions
 Search analytics

21. SpareBank1 – Live Demo
https://www2.sparebank1.no/

22. Komplett - Background
 Komplett NO, SE, DK… inWarhouse.se, MPX
 Existing Solr solution
 Mile long query with boosting per field
 Poor relevance
 Peripherals/accessories ranked higher than products
 Limited faceting
 No query completion or spellcheck
 Sloooooow indexing

23. Komplett - Requirements
 Superior and customisable relevance model
 Much more comprehensive indexing of products and specifications
 Spellcheck
 Query completion
 So much more faceting

24. Sébastien Muller
sebastien.muller@findwise.com