O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
State of Solr Security 2016
Ishan Chattopadhyaya
Engineer, Lucidworks
Typical Solr Deployments
Solr
Solr
Solr
Zookeeper
User
Application
22/10/16
History of Solr security
● "First and foremost, Solr does not concern
itself with security either at the document...
22/10/16
History of Solr security
● Servlet container based security
● SOLR-4470 patch for internode communication
22/10/16
What do we mean by security?
● Restricting access to trusted users
● Restricting trusted users to only allow acce...
22/10/16
SSL
● Introduced in Solr 4.2 (standalone), Solr 4.7 (cloud)
● Basic steps:
– Generate/obtain a certificate
– Conv...
22/10/16
Authentication framework
● Introduced in Solr 5.2 (SOLR-7274)
● Only supported with SolrCloud
● Out of the box im...
22/10/16
Kerberos authentication
● Introduced in Solr 5.2 (SOLR-7468)
● Based on hadoop-auth library
● Only supported with...
22/10/16
Kerberos authentication
● Basic steps:
– Choose service principals, client principals (e.g.
HTTP/<host>@REALM or ...
22/10/16
Kerberos: Delegation tokens
● Introduced in Solr 6.2
● Based on hadoop-auth library
● Reduce load on KDC
● Comple...
22/10/16
Basic authentication
● Introduced in Solr 5.3
● Provides an API endpoint to manage user credentials
● Salted pass...
22/10/16
Basic authentication
● Basic steps
– Setup ZK with security.json specifying (a) authc plugin as
BasicAuthPlugin, ...
22/10/16
PKI Authentication
● Introduced in Solr 5.3
● Used only for internode communication
● Based on public key infrast...
22/10/16
Custom authentication plugin
public class MyAuthcPlugin extends AuthenticationPlugin {
@Override
public void clos...
22/10/16
Authorization framework
● Introduced in Solr 5.2
● Only supported in SolrCloud
● Out of the box implementation:
–...
22/10/16
Rule-based Authorization plugin
● Introduced in Solr 5.3
● Supports users and roles
● Provides an API endpoint to...
22/10/16
Rule Based Authorization plugin
● Basic use:
– Adding user to a role:
curl --user solr:SolrRocks
http://localhost...
22/10/16
Ranger plugin
22/10/16
Ranger plugin
● Reference:
https://community.hortonworks.com/articles/15159/se
curing-solr-collections-with-range...
22/10/16
Custom authorization plugin
public class MyAuthzPlugin implements AuthorizationPlugin {
@Override
public void clo...
22/10/16
Custom authorization plugin
public abstract class AuthorizationContext {
public abstract SolrParams getParams() ;...
22/10/16
Storage level security
● Encrypting the index (LUCENE-6966, Renauld Delbru)
● Encrypting the index (Credeon/Hitac...
22/10/16
Zookeeper ACL
● Used to protect znodes created by Solr
● Permissions:
– CREATE, READ, WRITE, DELETE, ADMIN
● Out ...
22/10/16
Custom code
● Uploading JAR files
● Use config API to use request handlers from jar files
● -Denable.runtime.lib=...
22/10/16
Document and Field level security
● No out of the box support
22/10/16
General guidelines
● Plan security strategy early
● Use a firewall around Solr and Zookeeper
● Enable SSL
● Choos...
22/10/16
Future
● Better tools to configure a cluster for security
● More authorization plugins: document/field level secu...
Nächste SlideShare
Wird geladen in …5
×

State of Solr Security 2016: Presented by Ishan Chattopadhyaya, Lucidworks

566 Aufrufe

Veröffentlicht am

Presented at Lucene/Solr Revolution 2016

Veröffentlicht in: Technologie
0 Kommentare
1 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
566
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
6
Aktionen
Geteilt
0
Downloads
32
Kommentare
0
Gefällt mir
1
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

State of Solr Security 2016: Presented by Ishan Chattopadhyaya, Lucidworks

  1. 1. O C T O B E R 1 1 - 1 4 , 2 0 1 6 • B O S T O N , M A
  2. 2. State of Solr Security 2016 Ishan Chattopadhyaya Engineer, Lucidworks
  3. 3. Typical Solr Deployments Solr Solr Solr Zookeeper User Application
  4. 4. 22/10/16 History of Solr security ● "First and foremost, Solr does not concern itself with security either at the document level or the communication level. It is strongly recommended that the application server containing Solr be firewalled such the only clients with access to Solr are your own."
  5. 5. 22/10/16 History of Solr security ● Servlet container based security ● SOLR-4470 patch for internode communication
  6. 6. 22/10/16 What do we mean by security? ● Restricting access to trusted users ● Restricting trusted users to only allow access to certain set of operations/actions as per their role ● Security against eavesdroppers of network packets ● Document level security ● Field level security ● Storage level security ● Securing Zookeeper ● Remote code execution Solr Solr Solr Zookeeper User Application
  7. 7. 22/10/16 SSL ● Introduced in Solr 4.2 (standalone), Solr 4.7 (cloud) ● Basic steps: – Generate/obtain a certificate – Convert to PEM format using OpenSSL tools – Add the passwords, paths to keystore file to bin/solr.in.sh – Set a cluster property “urlScheme” to https in ZK – Start Solr ● Might need “haveged” on Vms ● ZooKeeper does not support SSL ● Reference: https://cwiki.apache.org/confluence/display/solr/Enabling+SSL
  8. 8. 22/10/16 Authentication framework ● Introduced in Solr 5.2 (SOLR-7274) ● Only supported with SolrCloud ● Out of the box implementations: – Kerberos authentication – Basic authentication
  9. 9. 22/10/16 Kerberos authentication ● Introduced in Solr 5.2 (SOLR-7468) ● Based on hadoop-auth library ● Only supported with SolrCloud ● Uses Kerberos authentication for internode communication ● Reference: https://cwiki.apache.org/confluence/display/solr/Kerber os+Authentication+Plugin
  10. 10. 22/10/16 Kerberos authentication ● Basic steps: – Choose service principals, client principals (e.g. HTTP/<host>@REALM or zookeeper/<host>@REALM or user@REALM) – Generate keytab files for all Solr, ZK nodes – Start ZK in Kerberized mode – Create a security.json file with authc plugin as KerberosPlugin – Create JAAS config files for every Solr host, specify their path in bin/solr.in.sh – Start Solr
  11. 11. 22/10/16 Kerberos: Delegation tokens ● Introduced in Solr 6.2 ● Based on hadoop-auth library ● Reduce load on KDC ● Complementary to Kerberos plugin – Supports operations: – RENEW, GET, CANCEL
  12. 12. 22/10/16 Basic authentication ● Introduced in Solr 5.3 ● Provides an API endpoint to manage user credentials ● Salted passwords stored in ZK ● Warning: (a) passwords are sent in cleartext, (b) /security.json in ZK must be write protected
  13. 13. 22/10/16 Basic authentication ● Basic steps – Setup ZK with security.json specifying (a) authc plugin as BasicAuthPlugin, (b) a default admin user/password hash – Start Solr – Use /admin/authentication endpoint to add/delete users curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json'-d '{"set-user": {"tom" : "TomIsCool", "harry":"HarrysSecret"}}'
  14. 14. 22/10/16 PKI Authentication ● Introduced in Solr 5.3 ● Used only for internode communication ● Based on public key infrastructure (shared + secret keys) ● Any authentication plugin can disable it: – implements HttpClientInterceptorPlugin
  15. 15. 22/10/16 Custom authentication plugin public class MyAuthcPlugin extends AuthenticationPlugin { @Override public void close() throws IOException {} @Override public void init(Map<String,Object> pluginConfig) {} @Override public boolean doAuthenticate(ServletRequest request, ServletResponse response, FilterChain filterChain) throws Exception { return false; } }
  16. 16. 22/10/16 Authorization framework ● Introduced in Solr 5.2 ● Only supported in SolrCloud ● Out of the box implementation: – RuleBasedAuthorizationPlugin
  17. 17. 22/10/16 Rule-based Authorization plugin ● Introduced in Solr 5.3 ● Supports users and roles ● Provides an API endpoint to manage users/roles ● Has preconfigured permissions: – security (security-read, security-edit), schema, config, core-admin, collection-admin, update, read, all ● Reference: https://cwiki.apache.org/confluence/display/solr/Rule-Bas ed+Authorization+Plugin
  18. 18. 22/10/16 Rule Based Authorization plugin ● Basic use: – Adding user to a role: curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization -H 'Content- type:application/json' -d '{ "set-user-role": {"tom": ["admin","dev"}}' – Adding permission for a role: curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization -H 'Content-type:application/json' -d '{"set- permission" : {"name":"update", "role":"dev"}}'
  19. 19. 22/10/16 Ranger plugin
  20. 20. 22/10/16 Ranger plugin ● Reference: https://community.hortonworks.com/articles/15159/se curing-solr-collections-with-ranger-kerberos.html ● Source: https://github.com/apache/incubator-ranger/tree/mas ter/ranger-solr-plugin-shim
  21. 21. 22/10/16 Custom authorization plugin public class MyAuthzPlugin implements AuthorizationPlugin { @Override public void close() throws IOException {} @Override public AuthorizationResponse authorize(AuthorizationContext context) { return null; } @Override public void init(Map<String,Object> initInfo) {} }
  22. 22. 22/10/16 Custom authorization plugin public abstract class AuthorizationContext { public abstract SolrParams getParams() ; public abstract Principal getUserPrincipal() ; public abstract String getHttpHeader(String header); public abstract Enumeration getHeaderNames(); public abstract String getRemoteAddr(); public abstract String getRemoteHost(); public abstract List<CollectionRequest> getCollectionRequests() ; public abstract RequestType getRequestType(); public abstract String getResource(); public abstract String getHttpMethod(); public enum RequestType {READ, WRITE, ADMIN, UNKNOWN} public abstract Object getHandler(); }
  23. 23. 22/10/16 Storage level security ● Encrypting the index (LUCENE-6966, Renauld Delbru) ● Encrypting the index (Credeon/Hitachi) [https://psg.hitachi- solutions.com/credeon/secure-full-text-search] ● Secure HDFS – Basic steps: ● bin/solr start -c -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs -Dsolr.hdfs.home=hdfs://host:port/path – Reference: https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+H DFS
  24. 24. 22/10/16 Zookeeper ACL ● Used to protect znodes created by Solr ● Permissions: – CREATE, READ, WRITE, DELETE, ADMIN ● Out of the box implementations: – VMParamsAllAndReadonlyDigestZkACLProvider ● Read only user ● User with full access
  25. 25. 22/10/16 Custom code ● Uploading JAR files ● Use config API to use request handlers from jar files ● -Denable.runtime.lib=true or sign your jar files ● Reference: http://home.apache.org/~ctargett/RefGuidePOC/jekyl l-full/adding-custom-plugins-in-solrcloud-mode.html
  26. 26. 22/10/16 Document and Field level security ● No out of the box support
  27. 27. 22/10/16 General guidelines ● Plan security strategy early ● Use a firewall around Solr and Zookeeper ● Enable SSL ● Choose authentication and authorization strategy ● Secure confidential data stored in ZK with ACLs
  28. 28. 22/10/16 Future ● Better tools to configure a cluster for security ● More authorization plugins: document/field level security, sentry integration (SOLR-9578, SENTRY-1478) ● Consider separating out authc/authz plugins from solr-core into separate module ● Remove dependency on httpclient ● Avoid ZK exposure (SOLR-9057) ● ZK should use SSL (SOLR-8342, ZOOKEEPER-235, Zookeeper 3.5.1-alpha) ● BasicAuth to support standalone more (SOLR-9481) ● ZK ACL passwords as startup params is insecure (SOLR-8756) ● Secure impersonation (SOLR-9324) ● Improve documentation ● New UI doesn't work with Kerberos (SOLR-9516) ● Improve test framework

×