Managing Certificates and TLS Endpoints

Managing Certificates and TLS
Endpoints
Charles Neill & Henry Yamauchi
Security Engineering
Rackspace Hosting

• Charles Neill & Henry Yamauchi of the Security
Engineering team at Rackspace
• Tasked with managing a large number of TLS/SSL
endpoints
• Since Heartbleed, everyone's eyes are on TLS/SSL
• Many headaches later, we now have a tool called
TLSManager to help us out! (More on that later)
www.rackspace.com 2
What's this all about?

• Wire encryption protocol
• Provides authentication, integrity and confidentiality
• Based on Public Key Infrastructure (PKI)
• SSL originally designed at Netscape for eCommerce in the early-mid 1990’s
– SSLv1 was never released, leaving us with SSLv2 and v3 + TLSv1.0, 1.1, and 1.2 today
• Widely deployed on a variety of devices (smart phones, laptops, servers,
appliances, etc.)
• Deceptively simple to get started with (but can be much harder to maintain)
www.rackspace.com 3
What is TLS/SSL?

• Time is fast approaching where TLS/SSL will be mandatory
• Browsers beginning to warn users about sites not using TLS/SSL
Firefox
Chrome
• Encryption overhead, cost are no longer valid excuses
– Initiatives like Let's Encrypt (letsencrypt.org) are aiming to provide a free SSL certificate
authority very soon to provide free certificates for all
• Despite becoming easier/cheaper to set up, TLS/SSL can't just be turned on
and left alone in most cases, as we'll see…
www.rackspace.com 4
TLS adoption is increasing

•Email
•IRC
•APIs and web services
•VPN
•Appliances (load balancers, proxies, etc)
•Smart phone apps
www.rackspace.com 5
TLS/SSL not just for web servers

•Certificate Authorities (CAs)
•Protocols
•Ciphers
•Key exchange
•Key sizes
•Hashes
•Random Number Generators (RNGs)
www.rackspace.com 6
TLS/SSL moving parts

• Multiple protocol versions (SSLv2, v3, TLSv1, v1.1, v1.2)
• Certificates expire at different times
• Server misconfiguration, multiple servers for single domain
• Outdated systems
• Large number of endpoints to keep track of
www.rackspace.com 7
Issues managing TLS/SSL:

• Intermediate certs expire
• Server certificates can accidentally get checked into
version control and have to be revoked
• Lax or compromised CAs may have to reissue their certs
(e.g. Comodo attack)
• Use of self-signed certificates
www.rackspace.com 8
Also:

• New SSL vulnerabilities are discovered (e.g. POODLE)
• Bugs in encryption software like OpenSSL or LibreSSL (e.g.
Heartbleed)
• Regulation compliance (e.g. PCI-DSS 3.1 will disallow
TLSv1 in 2016)
• SHA1 in certs will be flagged insecure in Jan 1st, 2017
(browser warnings now)
www.rackspace.com 9
Oh, and don't forget:

• When first connecting, client and server engage in negotiation
• Client initially sends the cipher suites it supports
• Server then (ideally) selects the strongest cipher that both supports
• Server should be configured with only strong ciphers
• There's a catch: the client essentially gets to decide what cipher is used by
controlling the ciphers that they send to the server. In some man-in-the-middle
attacks, the attacker can potentially downgrade ciphers in this way (e.g.
LOGJAM).
www.rackspace.com 12
TLS cipher negotiation

• Combination of encryption, MAC and key exchange
algorithms
• Available suites depend on the SSL/TLS version
• TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
^ ^ ^ ^
| | | |
| | | MAC hash algo
| | Symmetric key attrs (algo, key size, mode)
| Authentication algo
Key exchange algo
Cipher suites

Recommended Avoid
Key Exchange
ECDHE_ECDSA
DHE_RSA
RSA
NULL
*_anon
MAC Hash
SHA384
SHA256
NULL
MD5
SHA1
Ciphers
AES_256_GCM
AES_128_GCM
AES_256_CBC
AES_128_CBC
NULL
RC2_*
RC4_*
DES_*
IDEA_*
3DES_*
*_EXPORT_*
Cipher suite suggestions

15www.rackspace.com
Other configuration options

• Avoid DES, RC4, MD5, 3DES key length < 128, export ciphers
• Prefer AES, GCM, SHA2, TLSv1.2, OpenSSL >= 1.0.1p / 1.0.2d
• Not all permutations of these are supported (see: cipher suites)
• Sometimes you are dependent on vendor support (e.g. hardware load
balancers)
• Some servers deliberately configured with weak ciphers to support old
browsers (IE6 still in use)
• Cert expiration is just one reason to replace them (other reasons: use of SHA1,
weak key sizes, cert revocation, CA compromises, etc)
Server configuration

Example nginx configuration
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-
SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-
SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-
AES256-SHA;
ssl_prefer_server_ciphers on;
Example Apache configuration
SSLProtocol All -SSLv2 -SSLv3
SSLCipherSuite HIGH:!aNULL:!MD5
SSLHonorCipherOrder on
More in-depth examples can be found on https://cipherli.st/
Configuration examples

• Certificate ordered/installed has too short a key (< 1024 bits)
• Full certificate chain (CA bundle) not installed
• Preferred cipher not set correctly
• ‘Honor cipher order’ feature not turned on
• SSL engine/library outdated
• Server not sufficiently protected (so private key is not protected)
• Servers in load balance group configured inconsistently
Other server configuration issues

• Protects past conversations even if server private key is
compromised
• Uses ephemeral keys in key agreement protocols
• ECDHE_ECDSA, DHE_RSA
PFS (Perfect Forward Secrecy)

• Server configuration to only accept secure connections for this site
• Server initially sends Strict-Transport-Security header to browser
• Browser makes a note that site should only be accessed with HTTPS
• If an attempt is made to connect to port 80, force user to reconnect via
HTTPS (port 443 by default)
• Particularly useful if sites mix secure and unsecure resources
STS / HSTS

21www.rackspace.com
Vulnerabilities

• Heartbleed (leaks random memory in plain text. Fixed in OpenSSL 1.0.1g)
• Logjam (force export ciphers) / WeakDH (weak/reused Diffie-Hellman params)
• BEAST (browser issue but can be mitigated server side. TLSv1 and earlier)
• CRIME (TLSv1 compression)
• FREAK (export ciphers)
• POODLE (affects SSLv3)
Some recent vulnerabilities

• Problem: MITM can force client and server to agree to use a lower protocol
version than what they actually support
• Note: The above is possible because SSL/TLS handshake is in cleartext
(session key has not been agreed to yet)
• Solution: Browser include a dummy cipher to tell the server to check whether its
highest protocol version is greater than ClientHello’s. If so, abort
communication.
• Fake ciphers (x00FF)
– EMPTY-RENEGOTIATION-INFO-SCSV
– TLS_FALLBACK_SCSV
* Signaling Cipher Suite Value
TLS downgrade attack protection (SCSV*)

• Ciphers allowed by the US Government for the export market
• Devised in the 1990’s. Laws restricting encryption export are no longer
in force.
• TLS 1.2 no longer includes them
• Export ciphers use short keys (40-56 bit), making them insecure
• Identified by the presence of EXP or EXPORT in their name
Export ciphers

• Just because an endpoint is acceptable today does not
mean it will remain so tomorrow since:
–Certificates expire
–Servers are reconfigured
–New vulnerabilities are uncovered
–Increasing computing power can make old
standards/assumptions obsolete (e.g. SHA1)
• This means you need to be constantly monitoring your
endpoints
In summary

Constantly?!

28www.rackspace.com
How to keep up?!

• OpenSSL's "s_client" is useful for debugging SSL. On most
Linux/UNIX systems, you can call it like so:
openssl s_client [-debug] –host host.com –port 443
• The "-debug" flag will give you more information about the
actual connection, showing a hex dump of the handshake
and negotiation process
• Doesn't tell you much about vulnerabilities, but gives you
the information you need if you already know what you're
looking for
OpenSSL's s_client

From the testssl.sh site:
"testssl.sh is a free command line tool which checks a server's
service on any port for the support of TLS/SSL ciphers,
protocols as well as recent cryptographic flaws and more."
testssl.sh

• Reports SSL/TLS issues such as:
– SSL/TLS version
– Weak protocols, ciphers and hashes
– Known vulnerabilities
• Uses an easy to understand letter rating system
• Scoring is calculated by assigning weights to:
– Protocol support (30%)
– Key exchange support (30%)
– Cipher strength (40%)
• Can be accessed through API
– Official API client here: https://github.com/ssllabs/ssllabs-scan
Qualys SSL Labs

Score Meaning
A+ Domains with no warnings and use HSTS and TLS_FALLBACK_SCSV
A Domains with 2048 bit keys, at least TLS 1.2 support and no warnings
A- Domains with 2048 bit keys, at least TLS 1.2 support but with some warnings
B
Uses SSL 3.0, RC4, missing intermediate certificates, vulnerable to BEAST or
CRIME, NS >= 65
C Domain is vulnerable to POODLE, NS >= 50
D Numeric Score >= 35
E Numeric Score >= 20
M Domain and cert host mismatch
T Certificate is not trusted
F
Uses MD5, keys with length < 1024, or SSLv3 is the strongest protocol
supported
Qualys letter rating system

33www.rackspace.com
You want an A+, right?

• https://cipherli.st
• Collection of good configurations for TLS/SSL, covering
several different servers (Apache, nginx, HAProxy, Postfix,
OpenSSH …)
• Useful for making recommendations to product teams for
software you might not have configured before
• Simple copy-and-paste format takes headaches out of
implementers making decisions about certain ciphers or
other settings (though you should vet these configurations
based on what clients you want/need to support!)
Cipherli.st

• Hosts several endpoints with broken SSL configurations
• Great resource for vetting SSL verification code / test
scripts
• Examples:
–https://expired.badssl.com/
–https://self-signed.badssl.com/
• Many, many more!
badssl.com

Wanna manage all those endpoints manually?

One possible solution: TLSManager

• Automatically scans daily all endpoints in its database
• Uses Qualys Labs SSL tool (https://www.ssllabs.com/ssltest)
• Reports SSL vulnerabilities
• Proactively warns of any certificates about to expire
• Emails reports to interested parties
• Keeps a history of the results
Features

• On Github: https://github.com/rackerlabs/TLSManager
OR http://bit.ly/1MUQiss
• License: Apache 2.0
• Pull requests most welcome!
Go get it!

42www.rackspace.com
Current Events

Google + BoringSSL

Let's Encrypt

SHA-1 IS TOTES BROKEN! SAVE YOURSELF!

Diffie-Hellman APOCALYPSE!1!!

• SSL/TLS management is more involved than it looks at first
• Lots of moving parts
• A constantly changing landscape: a server rated A+ today can be
downgraded tomorrow if:
– Certs expire
– New cipher vulnerabilities are uncovered
– Bugs in SSL/TLS libraries are reported
– Server is misconfigured
– Cert or CA compromise
• It's helpful to monitor SSL endpoints in an automated way to keep from
missing any issues
Closing thoughts

Questions?
Charles Neill
Twitter: @ccneill
Github: cneill
Email: cneill@inventropy.us
Henry Yamauchi
Email: henry.yamauchi@rackspace.com

• Group that focuses on the security of one of the largest
open source projects in the world!
• IRC channel: #openstack-security on Freenode
• Weekly meetings: #openstack-meeting-alt @ 1700 UTC
Thursdays
• Mailing list: openstack-dev ML with [Security] tag
OpenStack Security Project

THANK YOU
RACKSPACE® | 1 FANATICAL PLACE, CITY OF WINDCREST | SAN ANTONIO, TX 78218
US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM
© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED S TATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

Managing Certificates and TLS Endpoints

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Recently uploaded

Recently uploaded (20)

Managing Certificates and TLS Endpoints