In the Juno summit, Symantec presented it's perspective on securing Keystone. Security is really a mindset and process. We proposed a layered security approach starting with the process for securing Keystone architecture, followed by securing the environment where Keystone is deployed and configured. Since then we have been implementing those security measures in our production environment. In this talk, we will discuss exactly how we have made our Keystone deployment secure and what we have learnt along the way.

1. Secure Keystone Deployment:
Lessons Learned and Best Practices
Priti Desai
Sr. Software Engineer
Secure Keystone Deployment 1

2. The Symantec Team
• Cloud Platform Engineering
– We are building a consolidated cloud platform that provides infrastructure and
platform services for next generation Symantec products and services
• Me
– In Security for over 6 years
– Symantec Insight - Reputation Based Security
– Symantec Data Analytics Platform
– OpenStack Engineer - Keystone
– OpenStack Security Group
– Cop Open Source
Secure Keystone Deployment
2

3. OpenStack Security Group
Secure Keystone Deployment
3
security
notes
Retrieved from http://www.openstack.orgRetrieved from http://docs.openstack.org

4. Secure Keystone Deployment
Why is Keystone security critical?
What is Keystone?
How is Authentication process implemented in
Keystone?
How is Authorization mechanism implemented in
OpenStack?

5. AuthN Overview
Secure Keystone Deployment
5
Cloud User
Cloud User
Identity
(SQL/LDAP)
Keystone
Token (SQL)
Identity
(SQL/LDAP)
Keystone
Token (SQL)
Request sent with
Username and Password
Verify username and
password (hash of
password)
Successful verification
Request metadata for user
tenant relationship
Assignment
(SQL)
Assignment
(SQL)
User tenant relationship
information
Request to generate new
token
Response with new token
Response with token

6. AuthZ Overview
Secure Keystone Deployment
6
Cloud User
Cloud User Keystone
OpenStack
Service
Keystone
OpenStack
Service
Request sent with session
token
Verify session token
Successful verification
Is this token correct?
Does it allow the service
usage?
Service executes
the request
Response with success

7. Secure Keystone Deployment
Why is Keystone security critical?
Does it store/transmit any sensitive information?
What kind of cloud asset does it store?
Is any type of attack possible on Keystone? Can it
bring down the entire cloud?

8. Keystone Security is Critical
Secure Keystone Deployment
8
• Gatekeeper
• Access to OpenStack Cloud
• Assets
• Users
• Passwords
• Tokens
• Roles
• Catalog
• Vulnerable to DoS
Retrieved from http://internet.phillipmartin.info
Retrieved from http://blogs.citypages.com
Retrieved from http://assets.nydailynews.com

9. What was our approach to identifying
key vulnerabilities?
Secure Keystone Deployment
9

10. Security Risks
Secure Keystone Deployment
10
• Global Security Office
 Threat Model
 Penetration Tests
 Traceability Matrix
Retrieved from http://www.technetics.com.au

11. Threat Model
Secure Keystone Deployment
11

12. Secure Keystone Deployment
12
Spoofing
Tampering
Repudiation
Information Disclosure
Denial of Service
Elevation of Privileges
Threat Model

13. What kind of security deficiencies did
we discover?
Secure Keystone Deployment
13

14. Secure Keystone Deployment
14
Attack: Keystone user credential theft
Attack: Insecure file permissions on
Keystone.conf
Keystone.conf
Attack: Access to cloud admin
privileges for almost free
Attack: Leaking sensitive data in log
messages
Attack: DoS – Authentication chaining
- Havana
Attack: Unauthorized access to MySQL
database
Many more …

15. Traceability Matrix
Secure Keystone Deployment
15
✖
✖
✖

16. Keystone User Credential Theft
Secure Keystone Deployment
16

17. Mitigate: Secure Communication - SSL
Secure Keystone Deployment
17
Hardware Load Balancer Hardware Load Balancer
Keystone KeystoneKeystone
SSL Client
SSL Server
SSL Client
SSL Server
mod_ssl
35357/SSL 5000/SSL
mod_ssl
35357/SSL 5000/SSL
mod_ssl
35357/SSL 5000/SSL
Public API Admin API

18. Insecure file permissions on Keystone.conf
Secure Keystone Deployment
18
Mitigate:
• Restrict ownership to service user
- chown keystone:keystone /etc/keystone/keystone.conf
• Restrict to read and write by the owner
- chmod 640 /etc/keystone/keystone.conf
hostnameabc
hostnameabc
hostnameabcuser
user
user

19. Access to admin privileges is almost free
Secure Keystone Deployment
19
• Service Token
• Bootstrap Keystone
• Cloud admin privileges
• Register bad service/endpoints

20. Mitigate: Disable Service Token
• Comment out admin_token from /etc/keystone/keystone.conf:
admin_token=e2112effd3ff05b8c88ad14e096e6615
• Remove admin token auth middleware from
/etc/keystone/keystone-paste.ini:
[filter:admin_token_auth]
paste.filter_factory =
keystone.middleware:AdminTokenAuthMiddleware.factory
Secure Keystone Deployment
20

21. Who is the cloud admin now?
Secure Keystone Deployment
21

22. Create Cloud Admin
• Leveraging Keystone Domain
• Before disabling service token:
• Create a domain “cloud_admin_domain”
• Grant “admin” role to appropriate user “Bob Smith”
• Update keystone policy.json file:
• Replace:
"cloud_admin": [["rule:admin_required", "domain_id:admin_domain_id"]],
• With:
"cloud_admin”: [["rule:admin_required”,"domain_id:<cloud_admin_domain_id>"]],
Secure Keystone Deployment
22

23. Leaking Sensitive Information in Log Messages
• Debug mode include plaintext request logging
• Passwords
• Tokens
• Mitigate:
• Disable debug mode in keystone.conf with:
• With debug mode ON, upgrade keystone client:
• python-keystoneclient >= 0.10.1 (OSSN-0024)
Secure Keystone Deployment
23
[DEFAULT]
debug=False

24. Leaking Sensitive Information in Log Messages
Identity API V2 - INFO level logs contains auth tokens (OSSN-0023)
Mitigate:
• Set the log level to WARN in logging.conf:
Secure Keystone Deployment
24
[handler_file]
class = FileHandler
Level = WARN

25. Keystone DoS Attack
Identity API V3 – Authentication Chaining – CVE-2014-2828
Secure Keystone Deployment
25

26. Keystone DoS Attack
Mitigate:
• Impacted Versions: from 2013.1 to 2013.2.3
• Patch applied during IceHouse rc2
• Upgrade Keystone >= 2013.2.4
Secure Keystone Deployment
26

27. Q&A
Let’s talk…
Secure Keystone Deployment 27

28. Thank You
Priti Desai
Priti_Desai@symantec.com
@pritidesai8

29. References
• http://docs.openstack.org/developer/keystone/
• https://blog-nkinder.rhcloud.com/?p=7
• https://blueprints.launchpad.net/keystone/+spec/service-scoped-
tokens
• http://docs.openstack.org/sec/
• http://www.florentflament.com/blog/setting-keystone-v3-
domains.html
• https://wiki.openstack.org/wiki/Security_Notes
Secure Keystone Deployment
29

30. References (Images)
• Crime Identity Theft: http://internet.phillipmartin.info/crime_identity_theft.gif
• Computer Theft: http://blogs.citypages.com/blotter/Computer%20theft.gif
• Mickey Washington ID:
http://assets.nydailynews.com/polopoly_fs/1.1864391!/img/httpImage/image.jpg_gen/de
rivatives/article_970/mickey13n-1-web.jpg
• Threat, Asset, and Vulnerability:
http://www.technetics.com.au/images/easyblog_images/79/b2ap3_thumbnail_manage_y
our_risk_400_20140924-122014_1.jpg
• Openstack security Notes: http://www.openstack.org/assets/openstack-logo/openstack-
one-color-alt.pdf
• OpenStack security Guide: http://docs.openstack.org/common/images/openstack-
security-guide.jpg
Secure Keystone Deployment
30