ipsec talk

1. 1
IP Security
(IPSec)
Thomas Lee
Chief Technologist –QA
thomas.lee@qa.com

2. 2
Agenda
 What is IPSec?
 How does IPSec Work?
 Configuring/Using IPSec
 Issues
 Best Practices
 Resources

3. 3
What is IPSec?
 Framework of open standards for ensuring private, secure
communications over Internet Protocol (IP) networks
 IPSec provides authenticated and encrypted traffic between hosts
at the IP protocol level
 Provides aggressive protection against private network and Internet
attacks through end-to-end security.
 Protects communication between workgroups, local area network
computers, domain clients and servers, branch offices (which
might be physically remote), extranets, and roving clients.
 IPSec is the long-term direction for secure networking

4. 4
IPSec Objectives
 To protect the contents of IP packets.
 To provide a defense against network attacks through
packet filtering
 To enforce trusted communication based on either local
or central policy
 These objectives are met through the use of
cryptography-based protection services, security
protocols, dynamic key management and Windows
Group Policy.

5. 5
Why IPSec?
 IPv4 not designed with security in mind
 Attacks possible with IPv4
• Eavesdropping
• Data modification
• Identity spoofing (IP address spoofing)
• Denial-of-service attack
• Man-in-the-middle attack
 These can be avoided by use of IPSec

6. 6
IPSec Protection
 Eavesdropping
• The Encapsulating Security Payload (ESP) protocol in IPSec provides data
confidentiality by encrypting the payload of IP packets
 Data modification
• Cryptography-based keys, shared only by the sending and receiving computers,
are use to create a cryptographic checksum for each IP packet.
• Modification of the data alters the checksum, which indicates to the receiving
computer that the packet was modified in transit
 Identity spoofing
• IPSec allows verification of identities without exposing that information to an
attacker.
• Mutual authentication establishes trust between the hosts.
 Man-in-the-middle attacks
• IPSec combines mutual authentication with shared, cryptography-based keys.
 Denial-of-service attacks
• IPSec uses IP packet filtering allow, secure, or block traffic based on IP address
ranges, IP protocols, or even specific TCP/UDP ports.

7. 7
Agenda
 What is IPSec?
 How does IPSec Work?
 Configuring/using IPSec
 Issues
 Best Practices
 Resources

8. 8
filtersfiltersfiltersfilters
How Components Interacts?
 Internet Key Exchange (IKE) - Identity Protect Mode – defined in RFC 2409
 Phase 1 “Main Mode” establishes IKE SA – trusted channel between systems, negotiation establishes
encrypted channel, mutual trust, and dynamically generates shared secret key (“master” key)
 Phase 2 “Quick Mode” establishes IPSec SAs – for data protection, one SA for each direction identified by
packet label (SPI), algorithms and packet formats agreed, generates shared “session” secret keys derived
from “master” key
NICNIC
TCPIPTCPIP
ApplicationApplication
Server or GatewayServer or Gateway
IPSecIPSec
DriverDriver
IPSecIPSec
PolicyAgentPolicyAgentIKE (ISAKMP)IKE (ISAKMP)
IPSecIPSec
DriverDriver
IPSecIPSec
PolicyPolicy
AgentAgent IKE (ISAKMP)IKE (ISAKMP)
NICNIC
TCPIPTCPIP
Application/ServiceApplication/Service
clientclient
““IKE Responder”IKE Responder”““IKE Initiator”IKE Initiator”
UDP port 500UDP port 500
negotiationnegotiation
1 IKE SA1 IKE SA
2 IPSec SAs2 IPSec SAs
IP protocol 50/51IP protocol 50/51

9. 9
IPSec Policy
 One Active IPSec Policy
• Multiple IPSec Policies can be defined
 Policy Consists of
• ISAKMP Policy
• IPSec rules
– An IPSec policy can have many rules
 IPSec Rules
• Filter – identifies the traffic to secure/drop/etc
• Filter action – drop, deny, authenticate, encrypt
• Authentication, encryption, etc

10. 10
IPSec Policy Components
 Polling interval used to detect changes
in policy
 IKE parameters, such as encryption key
lifetimes.
 IPSec behavior for the policy
 The types of traffic to which an action is
applied
 Permit, block, or secure
 Kerberos, certificate, or preshared key
 LAN, Dialup, or both

11. 11
IPSec packet filtering
 Filters allow and block traffic
 Filters can overlap
• Most specific match determines action
 NO stateful inspection
 Example: to open only port 80 on the IIS:
From IP To IP Protocol Src Port Dest Port Action
Any My Internet IP Any n/a n/a Block
Any My Internet IP TCP Any 80 Permit

12. 12
Negotiation of Protection
 Require two messages
• Initiator to Responder : (contains proposals)
• Responder to Initiator: (contains a selected proposal)
• Details later!
 Protection suites:
Attribute Attribute Value
Encryption algorithm DES, 3DES, Null
Integrity algorithm MD5, SHA-1, Null
Authentication method Kerberos, preshared key, certificate
Diffie-Hellman group Group 1 (768-bit), Group 2 (1024-bit)

13. 13
IPSec Modes
 Transport mode
• Used for IPSec peers doing end-to-end security
• Provides protection for upper-layer protocol data units
(PDUs)
 Tunnel mode
• Used by network routers to protect IP datagrams passing
across insecure network
• Provides protection for entire IP datagrams

14. 14
Security Associations
 Combination of mutually agreed security services, protection
mechanisms, and cryptographic keys
 ISAKMP SA
 IPSec SAs
• One for inbound traffic
• One for outbound traffic
 Security Parameters Index (SPI)
• Helps identify an SA
 Creating SAs
• Main Mode for ISAKMP SA
• Quick Mode for IPSec SAs

15. 15
Agenda
 What is IPSec?
 How does IPSec Work? (at the packet level!)
 Configuring/using IPSec
 Issues
 Best Practices
 Resources

16. 16
Internet Key Exchange
 How IPSec peers establish SAs
 Combines ISAKMP and the Oakley Key
Determination Protocol
• ISAKMP is used to identify and authenticate peers, manage
SAs, and exchange key material
• Oakley Key Determination Protocol is used to generate
secret key material for secure communications (Diffie-
Hellman key exchange algorithm)

17. 17
ISAKMP Message Structure
IP header ISAKMP payloads
UDP message
IP datagram
UDP
header
ISAKMP
header
ISAKMP uses UDP source/destination port 500

18. 18
ISAKMP Header
Initiator Cookie
Responder Cookie
Next Payload
Major Version
Minor Version
Exchange Type
Flags
Message ID
LengthA

19. 19
ISAKMP Payloads
 SA
 Proposal
 Transform
 Vendor ID
 Nonce
 Key Exchange
 Notification
 Delete
 Identification
 Hash
 Certificate Request
 Certificate
 Signature

20. 20
IPSec Headers
 IPSec Headers live inside IP datagrams and define
IPSec contents
 Authentication Header (AH)
• Provides data origin authentication, data integrity, and
replay protection for the entire IP datagram
 Encapsulating Security Payload (ESP)
• Provides data origin authentication, data integrity, replay
protection, and data confidentiality for the ESP-
encapsulated portion of the packet

21. 21
AH Transport Mode
IP Upper layer PDU
IP AH
Authenticated
Upper layer PDU

22. 22
AH Tunnel Mode
AH
Authenticated
IP
IP
IP (new)
Upper layer PDU
Upper layer PDU

23. 23
Next Header
Payload Length
Reserved
Security Parameters Index
Sequence Number
Authentication Data
Payload
. . .
Authentication Header (AH)
. . .

24. 24
ESP Transport Mode
IP ESP ESP
Auth
Data
Encrypted
Authenticated
IP Upper layer PDU
Upper layer PDU

25. 25
ESP with AH Transport Mode
IP ESP ESP
ESP
Auth
Encrypted
Authenticated with AH
IP
AH
Upper layer PDU
Upper layer PDU
Authenticated with ESP

26. 26
ESP Tunnel Mode
IP (new) ESP ESP
Auth
Data
IP
Encrypted
Authenticated
IP Upper layer PDU
Upper layer PDU

27. 27
Security Parameters Index
Sequence Number
Payload
Padding
Padding Length
Next Header
Authentication Data
. . .
. . .
. . .
ESP Header and Trailer

28. 28
Internet Key Exchange
 Standard that defines a mechanism to establish SAs
 Combines ISAKMP and the Oakley Key
Determination Protocol
• ISAKMP is used to identify and authenticate peers, manage
SAs, and exchange key material
• Oakley Key Determination Protocol is used to generate
secret key material for secure communications (Diffie-
Hellman key exchange algorithm)

29. 29
Main Mode Negotiation
 Phases of main mode negotiation:
1. Negotiation of protection suites
2. A Diffie-Hellman exchange
3. Authentication
 Six ISAKMP messages
• 1, 2 – all authentication types
• 3, 4, 5, and 6 - vary by Authentication type

30. 30
Authentication in MM Negotiation
 Kerberos Authentication
• Kerberos Tokens exchanged and validated
 Certificate Authentication
• Certificates and signatures exchanged and validated
 Preshared Key Authentication
• Hash payloads exchanged and validated

31. 31
Main Mode Negotiation Messages
 Message 1
• Sent by initiator
• Contains proposed security association details, vendor ID
 Message 2
• Sent by responder
• Contains acceptable SA proposal, vendor ID
 These messages negotiate:
• Encryption (DES, 3DES)
• Identity Algorithm (MD5, SHA-1)
• Authentication Method (Kerberos, Pre-shared key, Certificate)
• Diffie-Hellman group (768-bit, 1024-bit, 2048-bit)

32. 32
Main Mode – Kerberos Authentication
 Message 3 - Sent from initiator
• Contains key exchange, Nonce, initiator’s Kerberos Token, NAT
Discovery information
 Message 4 - Sent from responder
• Contains key exchange, Nonce, responder’s Kerberos Token ,
NAT Discovery information
 Message 5 - Sent from initiator (encrypted)
• Contains identification (of initiator), plus hash
 Message 6 - Sent from responder (encrypted)
• Contains identification (of responder), plus hash

33. 33
Main Mode – Certificate Authentication
 Message 3 - Sent from initiator
• Contains key exchange, Nonce, NAT Discovery information
 Message 4 - Sent from responder
• Contains key exchange, Nonce, NAT Discovery information
• Also contains certificate request (list of trusted root CAs)
 Message 5 - Sent from initiator (encrypted)
• Contains Initiator’s Certificate, signature
 Message 6 - Sent from responder (encrypted)
• Contains Responder’s certificate, signature

34. 34
Main Mode – Pre-Shared Key Authentication
 Message 3 - Sent from initiator
• Contains Key Exchange, Nonce, NAT Discovery
 Message 4 - Sent from responder
• Contains Key Exchange, Nonce, NAT Discovery
 Message 5 - Sent from initiator (encrypted)
• Contains Identification (of initiator), hash
 Message 6 - Sent from responder (encrypted)
• Contains Identification (of responder), hash

35. 35
Quick Mode Negotiation
 Four ISAKMP messages to determine traffic to be
secured and how it is secured
 Initiator and responder exchange:
• SA payloads (how to secure traffic)
• Identification payloads (the traffic to secure)

36. 36
IPSec On the WireIPSec On the Wire
OrOr
Fun With NetMon!Fun With NetMon!
Demo

37. 37
Agenda
 What is IPSec?
 How does IPSec Work?
 Configuring/Using IPSec
 Issues
 Best Practices
 Resources

38. 38
Issues with IPSec
 Need for machine certificates
 Interoperability
 Performance/Overhead
 NAT traversal

39. 39
Need for Machine Certificates/Keys
 IPSec is based on machine to machine communication
• User credentials not used
• You therefore need machine certificates
 For Kerberos
• Machine is the security principal
• Only works for Windows 2000/2003/XP
 For Certificate based authentication
• How to manage/deliver certificates

40. 40
Performance/Overhead
 IPSec incurs three sets of overhead:
• Startup
• Wire protocol overheads
• Speed of encryption
 Start up over-head
• Main Mode – 6 packets
• Quick Mode – 10 packets

41. 41
Wire Protocol Overhead
Mode Inbound Outbound Total Bytes
Normal IP Traffic 2,084,031 94,646,227 96,730,257
AH Traffic 2,700,627 96,174,275 98,874,902
ESP Traffic 2,867,099 96,941,132 99,808,231
AH Overhead 616,596 1,528,048 2,144,645
ESP Overhead 783,068 2,294,905 3,077,974
% AH Overhead 29.59% 1.61% 2.22%
% ESP Overhead 37.57% 2.42% 3.18%

42. 42
IPSec Hardware Acceleration
 IPSec per-packet hardware acceleration for 10/100 Ethernet
 Client/Svr cards retail circa $100
 3Com
• 3CR990B-97 - 10/100 UTP
• 3CR990B-FX-97 – 10/100 Fiber
• Wire Speed IPSec
• Max 75 SAs supported
• http://www.3com.com/other/pdfs/products/en/400833.pdf
- Or -
• http://tinyurl.com/3er3f
 Intel
• Intel®
PRO/100 S Desktop/Server
• http://www.intel.com/network/connectivity/resources/doc_library/documents/
pdf/intel_ipsec_final.pdf
- Or -
• http://tinyurl.com/37hcn

43. 43
XP IPSec Performance Improvements
 Doubled number of new SAs per minute
 Reliable delete handling in IKE
 Doubled packet filtering speed (throughput)
 Client LDAP retrieval of AD policy 5 times faster than Windows
2000
 Both Intel and 3Com 32bit x86 10/100Ethernet offload support
shipping in the box

44. 44
IPSec NAT-T
 Network Address Translators (NATs) invalidate
IPSec packet protections
 IPSec NAT Traversal (NAT-T):
• Encapsulates ESP-protected payloads with a UDP header
• Defines additional Main Mode payloads to detect IPSec
NAT-T-capable peers and whether either is behind a NAT
• Defines an additional Quick Mode payload to indicate
untranslated addresses
• Allows ESP-protected traffic to traverse a NAT

45. 45
Agenda
 What is IPSec?
 How does IPSec Work?
 Configuring/Using IPSec
 Issues
 Best Practices
 Resources

46. 46
Scripting
 NETSH –C IPSEC – with Windows Server 2003
 Netsh IPsec
 No dump command 
• It is included
• It does nothing!
 Help text has few examples
 Error messages generally totally unhelpful
 Lots of trial and error seems to be needed!

47. 47
Best Practices
 Establish an IP Security deployment plan
 Avoid Pre-shared keys
 Configuration of certificate requests
 Script, script, script