The Apache Knox Gateway is an extensible reverse proxy framework for securely exposing REST APIs and HTTP-based services at a perimeter. It provides out of the box support for several common Hadoop services, integration with enterprise authentication systems, and other useful features. Knox is not an alternative to Kerberos for core Hadoop authentication or a channel for high-volume data ingest/export. It has graduated from the Apache incubator and is included in Hortonworks Data Platform releases to simplify access, provide centralized control, and enable enterprise integration of Hadoop services.

1. © Hortonworks Inc. 2014
Securing Hadoop’s REST APIs
Apache Knox Gateway
Hadoop Summit 2014
Kevin Minder
Larry McCayhttp://knox.apache.org/
user (at) knox.apache.org
dev (at) knox.apache.org

2. © Hortonworks Inc. 2014
What is Apache Knox?
• The Apache Knox Gateway is…
• an extensible reverse proxy framework
• for securely exposing REST APIs and HTTP based services at a
perimeter
• out of the box it provides:
• support for several of the most common Hadoop services
• integration with enterprise authentication systems
• several other useful features

3. © Hortonworks Inc. 2014
What the Apache Knox Gateway isn’t
• Not an alternative to Kerberos for strong Hadoop core authentication
• Not a channel for high volume data ingest or export

4. © Hortonworks Inc. 2014
History and Status of the Apache Knox Gateway?
• 2013-02: Accepted into Apache Incubator
• 2013-04: Released 0.2.0
• 2013-10: Released 0.3.0
• 2014-02: Graduated to Apache TLP
• 2014-04: Released 0.4.0, Included in HDP 2.1

5. © Hortonworks Inc. 2014
Why Knox?
Simplified Access
• Kerberos encapsulation
• Extends API reach
• Single access point
• Multi-cluster support
• Single SSL certificate
Centralized Control
• Central REST API auditing
• Service-level authorization
• Alternative to SSH “edge node”
Enterprise Integration
• LDAP integration
• Active Directory integration
• SSO integration
• Apache Shiro extensibility
• Custom extensibility
Enhanced Security
• Protect network details
• Partial SSL for non-SSL services
• WebApp vulnerability filter

6. © Hortonworks Inc. 2014
Layers Of Hadoop Security
Perimeter Level Security
• Network Security (i.e. Firewalls)
• Apache Knox (i.e. Gateways)
Authentication
• Kerberos
• Delegation Tokens
OS Security
• File Permissions
• Process Isolation
Authorization
• MR ACLs
• HDFS Permissions
• HDFS ACLs
• HiveATZ-NG
• HBase ACLs
• Accumulo Label Security
• XA Security Policies
Data Protection
• Transport
• Storage

7. © Hortonworks Inc. 2014
REST API
Hadoop
Services
What does Perimeter Security really mean?
Gateway
REST API
Firewall
User
Firewall
required at
perimeter
(today)
Knox Gateway
controls all
Hadoop REST
API access
through firewall
Hadoop
cluster
mostly
unaffected
Firewall only
allows
connections
through specific
ports from Knox
host

8. © Hortonworks Inc. 2014
What REST APIs does Hadoop support?
Service URL Example
WebHDFS http://localhost:50070/webhdfs
WebHCat (aka Templeton) http://localhost:50111/templeton
Oozie http://localhost:11000/oozie
HBase (via Stargate) http://localhost:60080
Hive (HiveServer2) http://localhost:10001/cliservice
jdbc:hive2://localhost:10001/?hive.server2.transport.mode=http;hive.server2.thrif
t.http.path=cliservice

9. © Hortonworks Inc. 2014
Basic Knox Operation & Extensibility

10. © Hortonworks Inc. 2014
Authentication and Identity Propagation
1. REST API Request
2. HTTP Basic Auth Challenge
kminder:secret
3. Authenticate kminder:secret
knox
keytab
4. Authenticates as
knox via SPNego
(i.e. Kerberos)
5. REST API Request
doAs kminder
0. Configure
knox user to be
known as
trusted proxy
LDAP

11. © Hortonworks Inc. 2014
Scalability and Fault Tolerance
Hadoop
Apache HTTPD+mod_proxy_balancer
f5 BIG-IP
HAProxy
Knox Cluster
(no shared state)
Really any
traditional
web tier
load balancer

12. © Hortonworks Inc. 2014
Extensibility: Providers and Services
• Both are dynamically discovered on the class path via Java’s ServiceLoader
• Providers
• Add new features to the gateway that can be used by Services
• Typically result in one or more filters being added to one or more chains
• Services
• Add new endpoints to the gateway to expose a specific service
• Assemble filter chains to enable specific features via providers
• Includes providing configuration to providers
• For example URL rewrite rules
• Associates endpoints with filter chains

13. © Hortonworks Inc. 2014
Topology Files
• Describe the services that should be exposed for a specific cluster
• Found in <GATEWAY_HOME>/conf/topologies
• Name of topology file dictates URL component
• sandbox.xml -> http://localhost:8443/gateway/sandbox/webhdfs/…
<topology>
<gateway>
<provider>
<role>authentication</role>
<name>custom</name>
</provider>
</gateway>
<service>
<role>WEBHDFS</role>
<url>http://localhost:50070</url>
</service>
</topology>
Location of
WebHDFS in
target cluster
Selects an
authentication
provider
implementation

14. © Hortonworks Inc. 2014
Enhanced Security

15. © Hortonworks Inc. 2014
Topology Leakage: WebHDFS Example
• WebHDFS direct
curl -i -X PUT 'http://localhost:50070/webhdfs/v1/user/guest/file1?op=CREATE&user.name=guest’
HTTP/1.1 307 TEMPORARY_REDIRECT
Location:
http://sandbox.hortonworks.com:50075/webhdfs/v1/user/guest/file1?op=CREATE&user.name=guest&namenoderp
caddress=sandbox.hortonworks.com:8020&overwrite=false
• WebHDFS via Knox
curl -u guest:guest-password -i -k -X PUT 'https://localhost:8443/webhdfs/v1/user/guest/file2?op=CREATE’
HTTP/1.1 307 Temporary Redirect
Location:
https://localhost:8443/gateway/sandbox/webhdfs/data/v1/webhdfs/v1/user/guest/file2?_=AAAACAAAABAAAACAg
UDT7-QQZlpkcm09lxrxI0Bgo9d-
Egghp_qxmd4pQsmm3zvYc3M_LrDBQpMBNA48DnMS9QOhyzywCMl1WAShyX4RUETPjEcZa6x9Jwz7TMANj
SRKMR6F3rKf93ME-VsI2Phe8CX72L6oiI778--8F9DQCO8LHFHzLL70iB13Hm2BLyj-x9p3tn7FOHxkbPl5d-
eHxVop7Dk
RPC and
HTTP address
of DataNode is
leaked
unnecessarily
to REST client
Encrypted query param contains
dispatch information used by gateway
when redirect followed

16. © Hortonworks Inc. 2014
Topology Leakage: Oozie Example
• Oozie direct
<configuration>
<property>
<name>oozie.wf.application.path</name>
<value>hdfs://foo:9000/user/bansalm/myapp/</value>
</property>
...
</configuration>
• Oozie via Knox
<configuration>
<property>
<name>oozie.wf.application.path</name>
<value>/user/bansalm/myapp/</value>
</property>
...
</configuration>
• Example of submitting an Oozie job from Apache docs
• https://oozie.apache.org/docs/4.0.1/WebServicesAPI.html
• HTTP POST XML below to /oozie/v1/jobs
REST client
must know
RPC address
of NameNode

17. © Hortonworks Inc. 2014
Partial SSL for non-SSL enabled services
REST API REST API
WebHCat
DMZ
Desktop
Gateway
HTTPS HTTP
First “hop”
through
public/corp
networks
protected with
SSL
Last “hop”
within
secure
network
non-SSL

18. © Hortonworks Inc. 2014
WebApp Vulnerability Filter
• The Knox WebAppSec provider allows for the plugin of vulnerability prevention filters
• Cross Site Request Forgery CSRF is currently provided
• Uses common required header technique
• Later releases will include more filters based on standard techniques
<provider
<role>webappsec</role>
<name>WebAppSec</name>
<enabled>true</enabled>
<param><name>csrf.enabled</name><value>true</value></param>
<param><name>csrf.customHeader</name><value>X-XSRF-Header</value></param>
<param><name>csrf.methodsToIgnore</name><value>GET,OPTIONS,HEAD</value></param>
</provider>

19. © Hortonworks Inc. 2014
Simplified Access

20. © Hortonworks Inc. 2014
Knox Service URLs vs. direct URLs
Service Direct URL Knox URL
WebHDFS http://namenode-host:50070/webhdfs https://knox-host:8443/webhdfs
WebHCat http://webhcat-host:50111/templeton https://knox-host:8443/templeton
Oozie http://ooziehost:11000/oozie https://knox-host:8443/oozie
HBase http://hbasehost:60080 https://knox-host:8443/hbase
Hive http://hivehost:10001/cliservice https://knox-host:8443/hive
Masters could
be on many
different hosts
One hosts,
one port
Consistent
paths

21. © Hortonworks Inc. 2014
Hadoop CLIs require full server configs
/etc/hive/conf/hive-site.xml
<property>
<name>hive.server2.thrift.http.port</name>
<value>10001</value>
</property>
<property>
<name>hive.server2.thrift.http.path</name>
<value>cliservice</value>
</property>
/etc/hadoop/conf/core-site.xml
<property>
<name>fs.defaultFS</name>
<value>hdfs://sandbox.hortonworks.com:8020</value>
</property>
/etc/hadoop/conf/hdfs-site.xml
<property>
<name>dfs.namenode.http-address</name>
<value>sandbox.hortonworks.com:50070</value>
</property>
/etc/hadoop/conf/yarn-site.xml
<property>
<name>yarn.resourcemanager.address</name>
<value>sandbox.hortonworks.com:8050</value>
</property>
/etc/hive-webhcat/conf/webhcat-site.xml
<property>
<name>templeton.port</name>
<value>50111</value>
</property>
/etc/oozie/conf/oozie-site.xml
<property>
<name>oozie.base.url</name>
<value>http://sandbox.hortonworks.com:11000/oozie</value>
</property>
HBase – Command line
These files
may all be
on different
nodes on
the cluster
too!

22. © Hortonworks Inc. 2014
Kerberos Encapsulation
1. REST API Request
2. HTTP Basic Auth Challenge
kminder:secret
3. Authenticate kminder:secret
knox
keytab
4. Authenticates as
knox via SPNego
(i.e. Kerberos)
5. REST API Request
doAs kminder
0. Configure
knox as trusted
proxy
The client isn’t
even aware the
cluster is secured
with Kerberos

23. © Hortonworks Inc. 2014
REST API REST API
Hadoop
REST API Reach: Intranet Access Model
DMZ
Desktop
Gateway
Users will
discover novel
ways to use easily
accessible REST
APIs

24. © Hortonworks Inc. 2014
HTML/JS REST
Hadoop
REST API Reach: Middleware Access Model
Web Tier / DMZ
Browser
“Give the APIs to the Apps”
GatewayApp
Server
REST
Most enterprises
cannot deal with
Kerberos in the
web tier and don’t
have CLI access

25. © Hortonworks Inc. 2014
REST API REST API
Hadoop
REST API Reach: Internet Access Model
DMZ
“Give the APIs to the Everyone”
Gateway
Internet
HaaS vendors
are exposing
Hadoop REST
APIs to the
internet. What
does the API tell
these clients to
know about your
cluster?

26. © Hortonworks Inc. 2014
Multi-Cluster Support
Gateway
http://knox:8443/gateway/green/webhdfs/v1 http://knox:8443/gateway/blue/webhdfs/v1
green
Production
Cluster
blue
Research
Cluster
One hosts,
one port for
many
clusters

27. © Hortonworks Inc. 2014
Simplified Client Certificate Management
hdfs
cert
hive
cert
hbase
cert
knox
cert
knox
pubkey
hive
pubkey
hbase
pubkey
hdfs
pubkey
• User only needs to trust Knox’s cert
• Admin only needs to manage multiple keys on Knox hosts

28. © Hortonworks Inc. 2014
Centralized Control

29. © Hortonworks Inc. 2014
SCP/SSHLogin Hadoop CLIs
Hadoop
Client Edge Node CLI Access Model
DMZ
Edge Node
Desktop
“Take the Users to the CLI”Limited
auditing on
edge node
CLI too hard
to install on
desktops

30. © Hortonworks Inc. 2014
REST APILogin REST API
Hadoop
Improved auditing and access control
DMZ
Desktop
Gateway
All activity
audited
consistently
Additional
authorization
control
available

31. © Hortonworks Inc. 2014
Service Level Authorization
• Control access to services by user, group or IP address
<provider>
<role>authorization</role>
<name>AclsAuthz</name>
<enabled>true</enabled>
<param>
<name>WEBHDFS.acl</name>
<value>*;admin;127.0.0.1</value>
</param>
</provider>

32. © Hortonworks Inc. 2014
XA Secure Integration
1. REST API Request
0. Distribute
policy
3. REST API Request
Policy Server
Agent
2. Service level
authorization decision
Agent
integrated as
authorization
provider
Policies
authored in
the portal and
distributed by
the policy
server

33. © Hortonworks Inc. 2014
KNOX-250: SSH Bastion Auditing Functionality
• Community is developing an extension
• Based on Apache MINA SSHD
• Provides administrative SSH access via Knox
• Further centralizes auditing of cluster administration
• https://issues.apache.org/jira/browse/KNOX-250

34. © Hortonworks Inc. 2014
KNOX-250: SSH Bastion Auditing Functionality
SSHLogin Hadoop CLI
Hadoop
DMZ
Desktop
Gateway
All activity
audited
consistently

35. © Hortonworks Inc. 2014
Enterprise Integration

36. © Hortonworks Inc. 2014
Apache Shiro Authentication Provider
• Apache Shiro is the primary authentication provider for Knox
• Used for both LDAP and Active Directory
• Apache Shiro is a popular JEE and JSE security framework
• Very modular and flexible architecture
• Many community extensions
• Integrated into Knox as a servlet filter

37. © Hortonworks Inc. 2014
Apache Shiro Authentication Provider
<provider>
<role>authentication</role>
<name>ShiroProvider</name>
<enabled>true</enabled>
<param>
<name>main.ldapRealm</name>
<value>org.apache.shiro.realm.ldap.JndiLdapRealm</value>
</param>
<param>
<name>main.ldapRealm.userDnTemplate</name>
<value>uid={0},ou=people,dc=hadoop,dc=apache,dc=org</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.url</name>
<value>ldap://localhost:33389</value>
</param>
<param>
<name>main.ldapRealm.contextFactory.authenticationMechanism</name>
<value>simple</value>
</param>
<param>
<name>urls./**</name>
<value>authcBasic</value>
</param>
</provider>

38. © Hortonworks Inc. 2014
SSO Integration
• Similar in concept Hadoop’s trusted proxy model
• Preconfigured for SiteMinder use case
• HTTP Headers used to propagate pre-authenticated user and group info
• Only acceptable for use in a tightly controlled network environment
<provider>
<role>federation</role>
<name>HeaderPreAuth</name>
<enabled>true</enabled>
<param>
<name>preauth.validation.method</name>
<value>preauth.ip.validation</value>
</param>
<param>
<name>preauth.ip.addresses</name>
<value>127.0.*</value>
</param>
</provider>

39. © Hortonworks Inc. 2014
OAuth 2
• OAuth is becoming the defacto standard for communicating a user’s
identity to REST APIs
• It allows for explicit authorization by the user for the application to
access resources
• It has a number of ways to represent the user and authentication
information to go over the wire
• JSON Web Token (JWT) is an emerging standard for representing the
various claims, attributes and scopes of an identity
• Can be used as a bearer token, URL parameter or Header
• OAuth is also gaining popularity as a federation token for SSO
integrations

40. © Hortonworks Inc. 2014
KNOX-393: OAuth Resource Provider
• Community investigating OAuth Federation Provider extension
• Considering Apache Oltu
• Warning: Diagram dramatically oversimplified
• There are a number of other potential flows
2. REST API Request
Authorization: Bearer <token>
3. validateAccessToken(<token>)
4. Authenticates as
knox via SPNego
(i.e. Kerberos)
5. REST API Request
doAs kminder
0. Configure
knox user to be
known as
trusted proxy
1. requestAccessToken(JWT)
return Bearer token
kminder

41. © Hortonworks Inc. 2014
What is next for Knox?
Jira Assignee Description
KNOX-393: OAuth Resource Provider for
Middleware and Application Integration
COMMUNITY OAuth 2 federation provider potentially based on Apache
Oltu for external application SSO to Knox and Hadoop
KNOX-355: Support Knox Authentication
Provider based on Hadoop Auth Module
(SPNEGO)
KNOX Team SPNEGO authentication support for Knox clients
KNOX-250: SSH Bastion Auditing Functionality COMMUNITY SSH tunneling and auditing functionality in addition to
REST gateway services.
KNOX-353: Support Hadoop Java Client URLs KNOX Team In order to be used Hadoop CLIs that can use REST, we
need to support the expected URLs. This is in addition to
the extended URLs for multiple Hadoop cluster support
by Knox.
KNOX-242: LDAP Authentication
Enhancements
KNOX Team Search attribute based authentication rather than simple
LDAP bind.
KNOX-74: Support YARN REST API KNOX Team Add support for the YARN REST API
KNOX-66: Support Ambari REST API access
via the Gateway
KNOX Team Add support for the Ambari REST API
TBD TBD What is important to you?

42. © Hortonworks Inc. 2014
Interested?
• We’re hiring!
• http://hortonworks.com/careers/open-positions/
• Especially hands on platform level development experience with
• Kerberos
• LDAP
• OAuth
• SAML
• JAAS/GSS-API
• Crypto