1.
Closing the TLS Authentication Gap<br />Marsh Ray<br />Steve Dispensa<br />PhoneFactor<br />www.phonefactor.com<br />

2.
Who are we, anyway?<br />

3.
We’re going to tell a story<br />Finding the flaw<br />Deciding how to address it<br />Private disclosure<br />Public disclosure<br />Post-disclosure work<br />Lessons learned<br />

4.
Finding the problem<br />August 11, 2009<br />

5.
So how did this happen?<br />It’s Microsoft’s fault!<br />Answered a question in a forum…<br />Which turned into a series of interesting discussions over the summer about MitM<br />Eventually, Marsh got fed up and went spelunking in mod_ssl<br />

6.
/* To allow authentication to complete in this auth hook, the <br /> * solution used here is to fill a (bounded) buffer with the <br /> * request body, and then to reinject that request body later.<br /> */<br />if (renegotiate && !renegotiate_quick && <br />(apr_table_get(r->headers_in, "transfer-encoding") ||<br />(apr_table_get(r->headers_in, "content-length") && <br />strcmp(apr_table_get(r->headers_in, "content-length"),<br />"0"))) && <br />!r->expecting_100) <br />{<br />intrv; <br />/* Fill the I/O buffer with the request body if possible. */ <br />rv = ssl_io_buffer_fill(r);<br />...<br />

7.
Apache 2.2 mod_ssl documentation<br />

8.
That can’t be right…<br />Buffering and replaying a request seemed… scary<br />So, I decided to make sure authentication continuity was maintained across the renegotiation.<br />Imagine my surprise when I couldn’t find a clear answer.<br />

9.
7.4.1. Hello Messages<br /> ...<br /> compression algorithms are initialized to null. The current<br /> connection state is used for renegotiation messages.<br />7.4.1.2. Client Hello<br /> When this message will be sent:<br /> When a client first connects to a server, it is required to send<br /> the ClientHello as its first message. The client can also send a<br /> ClientHello in response to a HelloRequest or on its own initiative<br /> in order to renegotiate the security parameters in an existing<br /> connection.<br />RFC 5246: Strangely quiet on renegotiation<br />

10.
So, Marsh disappeared for a couple of weeks, and out came:<br />

11.

12.
Demo!<br />

13.
We immediately understood the significance.<br />

14.

15.
Disclosure without disclosure, on September 8<br />

16.
Three attacks<br />Client certificate-based attack<br />Client certificates can trigger renegotiation<br />Upgrade attack<br />Different-strength crypto requirements can lead to renegotiation<br />Client-initiated attack<br />In theory, a client could start a renegotiation at any time<br />

17.
But wait, there’s more!<br />Browsers don’t always validate the server cert before handing out the client cert!<br />Therefore, a client cert can effectively be forwarded to any server on the net that accepts it<br />Browsers don’t always prompt for client certificates when they make can make an “intelligent” choice<br />Victim never knows what hit him<br />

18.
OK, does it matter?<br />We struggled to assess scope and impact…<br />Client certificate – first finding; mitigation painful<br />Upgrade attack – cute, but… meh…<br />Client-initiated – almost an afterthought at this point<br />Not absolutely sure it would even work<br />

19.
Is Renegotiation worth saving?<br />Disabling renegotiation completely would be:<br />Easy<br />Effective<br />Solve about 95% of the problem<br />Will it ever come back?<br />IP Source Routing, anyone?<br />

20.
Some uses of TLS renegotiation<br />

21.
Some uses of TLS renegotiation<br />Wikipedia<br />

22.
Wild speculation!<br />

23.
DoD Common Access Card System<br />3.5M active cards<br />1M card readers<br />Primarily based on client certificates!<br />Wikipedia<br />

24.

25.

26.

27.
Doesn’t this make you want a National ID card?<br />

28.
We realized:<br />Needed a coordinated effort to fix<br />Huge leak potential<br />We wanted near-simultaneous disclosure<br />Wanted a solution before the bug leaked<br />

29.
Project Mogul<br />September 14, 2009<br />

30.
Oh, by the way, it’s not this guy:<br />

31.
Disclosure plan<br />Decided on a phased disclosure plan:<br />Disclose a few respected security gurus<br />Disclose to SSL code owners and start a fix<br />Widen the circle carefully over time<br />Hope for a controlled public disclosure<br />

32.
The NDA<br />Everyone told us we were insane to want an NDA, until…<br />…they heard about the flaw!<br />We wanted pressure on vendors<br />Intentionally written to expire on 1/31/2010.<br />

33.

34.
"Both the insider and his friend were active members of the hacking group, and regularly attended the organization’s meetings. They used IRC channels to communicate back and forth with one another and relay information under assumed hacker names in an attempt to mask their identities."<br />

35.
First, we asked Frank Heidt of Leviathan Security,<br />Who confirmed our intuition about the impact of this vuln:<br />

36.

37.
Extraordinary claims require extraordinary proof<br />Ponytail immediately began flapping wildly. Took up smoking again.<br />Frank referred us to lots of helpful people, including:<br />Jon Callas, as an independent security review<br />Ben Laurie, for obvious reasons<br />Dan Geer, Kerberos<br />Jennifer Granick @ EFF<br />

38.
We thought we needed a plan. Turns out, we needed several.<br />Plan A:<br />Get code owners together and tell them all at once, under NDAs<br />Drawback:<br />Needed people besides coders<br />

39.
Plan B:<br />Get programmers and limited support people to Mountain View and disclose all at once under NDA<br />Drawbacks:<br />Vendors needed to know the bug before committing<br />Vendors needed some time to assess impact in order to figure out who needed to be involved<br />

40.
Plan C:<br />Disclose bug to code owners and limited support personnel under NDA and then go to Mountain View to work out the details<br />Drawback:<br />Had trouble getting some companies under NDA<br />

41.
Plan D:<br />Disclose in advance<br />to the fewest people possible (coders, PSIRT managers, …) <br />under group NDAs such as ICASI and Google <br />then get people to Mountain View to work out the details in a week or two<br />

42.
Disclosure<br />All disclosures were completed within about a week<br />Disclosed to Ben Laurie<br />Reproduced across the Internet, tempting the demo gods<br />Tried to disclose to IBM: NDA fail<br />Disclosed to Microsoft next<br />

43.
ICASI<br />Pointed to ICASI by Frank, IBM, and Microsoft<br />Disclosed to Steve Manzuik of Juniper/ICASI, leading to:<br />Microsoft<br />Intel<br />Cisco<br />Juniper<br />Nokia<br />IBM<br />

44.
Exceptions<br />There were a few notable exceptions:<br />Red Hat lawyers worked the weekend!<br />Sun : “Type your vuln here and hit submit ok thx bye”<br />Apple: We didn’t realize they had their own TLS code<br />Others, due to an attempt to limit scope<br />

45.
Mogul meeting: September 28, 2009<br />About 45 people representing about a dozen organizations<br />Description and captures, again<br />Severity and impact<br />Lots of time spent on client-initiated renegotiation<br />Solution discussion<br />Rescorla, Oskov, and Dispensa/Ray had identical proposals<br />

46.
Proposed solution<br />The obvious solution was to bind the cryptographic state from the previous handshake to the current one<br />This is easy:<br />Resend the verify_data from the previous Finished message<br />Already cryptographically secure<br />Already under consideration as a “channel binding”<br />Not a perfect solution, however:<br />Requires a TLS extension<br />Requires additional storage (bad for silicon?)<br />

47.
Post-conference work<br />Turns out it’s hard to organize a private, cross-vendor, ad-hoc team!<br />Manzuik requisitioned help from Paul Vixie / OpSecTrust<br />Manzuik set up [mogul-private] and a PGP keyring<br />We set up a private SILC channel<br />Good initial discussion on the lists, but vendor engagement dropped off quickly<br />No data!<br />

48.
Initial implementations of safe renegotiation<br />Nasko Oskov from Microsoft had a working implementation quickly<br />Eric Rescorla provided code for OpenSSL<br />Dispensa worked up a patch for GNUTLS<br />We suspect others were making progress<br />

49.
TLA<br />[this page intentionally left blank.]<br />

50.
Timeline tension<br />Work was going really slowly<br /> January 31 “couldn’t possibly work”<br />Not a Patch Tuesday<br />Not a weekday<br />BlackHat / ShmooCon<br />By late October, Steve was on repeated ICASI calls<br />Insisting that we postpone publishing<br />Our position was unchanged officially<br />Meanwhile, Marsh and Steve started to argue about scenarios<br />

51.
Public Disclosure<br />November 4, 2009<br />

52.
So, we were all minding our own business, when…<br />

53.
To: tls at ietf.org<br />Subject: [TLS] MITM attack on delayed TLS-client auth through renegotiation<br />From: Martin Rex <Martin.Rex at sap.com><br />Date: Wed, 4 Nov 2009 18:28:00 +0100 (MET)<br />After elaborating so much about the client cert authentication through renegotiation with Microsoft IIS, I'm beginning to believe that there is a potential security problem with that scheme, because it is susceptible to a MITM attack. <br />...<br />[TLS] turns into Full-Disclosure<br />

54.
Hilarity ensues<br />Dispensa calls Martin Rex<br />Project Mogul is notified in a very tense call<br />Vendors Hope It Goes Away™<br />Vendors end by insisting that it would be “extremely irresponsible” to publish<br />After all, “nobody will notice.”<br />

55.
So, did anyone notice?<br />

56.
Three hours after Martin Rex’s e-mail…<br />

57.
…it was re-tweeted a few times, too.<br />

58.
Steve gets bored and decides to do something else.<br />

59.
Any guesses as to how long it took for working exploit code to be posted to [full-disclosure]?<br />

60.
18 hours. 34 minutes.<br />

61.
Initial reactions were… mixed.<br />“The sky is not falling” –Moxie Marlinspike<br />It’s just like CSRF! (Whew! … Whew?)<br />“Most, if not all, major web applications have implementation level protections against CSRF… Those protection measures are effective against this new SSL man in the middle attack. Therefore, this vulnerability has minimal security impact for most websites and Internet users.” –Tom Cross, IBM ISS<br />

62.
A couple of days later…<br />

63.

64.

65.
Coming to terms with the bug<br />Yeah, but who cares?<br />The 41% of users who use the same password for Twitter as they use for… everything!<br />More importantly - you can’t tell what will be broken<br />In the end, the confusion was our fault<br />

66.
Post-disclosure Work<br />

67.
IETF<br />ID ready day 1<br />Flawed: undercounted SSLv3<br />No extensions!<br />Post-disclosure, Benn Bollay from F5 shared data: 22%!<br />Tons of e-mails on the IETF list<br />Practically a full-time job for Marsh<br />Finally, we added SCSV to address old servers<br />RFC 5746 very soon!<br />

68.

69.
Patch status<br />Several vendors have disabled renegotiation<br />A few vendors and projects have implemented the new RFC <br /><ul><li>www.phonefactor.com/sslgap</li></li></ul><li>So far, one commercial vendor has shipped:<br />

70.
OPERA FTW!!!<br />

71.
Not everyone has rolled out a fix<br />

72.
Some lessons learned<br />

73.
Security bugs are a no-win situation<br />Traumatic for vendors<br />Not great for researchers<br />Worst, of course, for the users<br />This was a really hard process – hard to balance lots of competing interests<br />

74.
There are other (bigger?) problems with SSL<br />PKI is great in theory, but:<br />~200 trusted root certificates in Firefox – do you trust them all?<br />There will never be a solution to the dancing bunnies problem<br />Applies to Business Bunnies too!<br />Sometimes, root CA’s do this:<br />

75.

76.
We needed hard data<br />We had no success getting vendors to contribute data <br />Would have been extremely helpful to know about SSLv3 prevalence before the IETF process<br />Does client-initiated renegotiation ever happen?<br />

77.
IETF security process?<br />It has been suggested that the IETF security review process is broken.<br />If it is, this bug isn’t why:<br />SSL was a Netscape creation<br />SSLv3 was utterly ownerless for years<br />The IETF did find it, a few months after us<br />The IETF could have done a better job adopting SSL<br />

78.
One last lesson<br />This one goes out to the slow/no-disclosure crowd with our compliments:<br />

79.

80.
So Marsh emailed Pavel Kankovsky and:<br />"I had some free time during the last days of 2006 and wrote the PoC exploit to carry out an experimental verification of the vulnerability. It was easier than I had expected because I found a clever way to makeOpenSSLcooperate. <br />“The exploit was finished on January 3, 2007."<br />

81.
One last question for you<br />Did we achieve our goal of minimizing the world’s exposure to the bug?<br />

82.
Questions?<br />marsh@extendedsubset.com<br />dispensa@phonefactor.com<br />www.phonefactor.com<br />mogul on silc.hick.org<br />mogul-open@lists.links.org<br />