IPSec VPN Cheat Sheet

Technology Consulting Services

Latest Articles

Written by David Torre

IPsec can use two different protocols – ESP or AH

ESP – Encapsulating Security Payload. Provides confidentiality, integrity, and authentication. Entire packet is encrypted. ESP is protocol number 50.
AH – Authentication Header. Provides only authentication and integrity; no confidentiality. Cannot pass through a NAT device when in transport mode. AH is protocol number 51.

IPsec operates in two different modes, including:

Tunnel Mode – Entire packet is encrypted, and a new ESP header (and footer) is added.
Transport Mode – Only the original payload is encrypted, leaving the original IP headers intact.

SAs: Security Associations

Generated after successful IKE exchange. An SA is a combination of keys and polices used to protect data between to VPN peers. SAs have the following attributes:

SPI – Security Parameter Index, used to uniquely identify SAs.
Destination IP Address
Security Protocol (ESP or AH)

SAs are established bidirectionally in Phase-1, and unidirectionally in Phase-2. They have a lifetime which can be defined in terms of either elapsed time and/or transferred data. SAs are managed using the Internet Security Association Key Management Protocol, or “ISAKMP.”

Proxy ID – Used to identify which SAs and subnets belong to which VPN. Be sure that your Proxy IDs (and subnets) match on both the local and remote firewalls; especially if more than one subnet resides behind one of the VPN gateways.

IPSec Phases

IPsec occurs in two phases. During Phase-1, a secure channel is established between two VPN gateways. During Phase-2, actual VPN tunnels are established. Although a single Phase-1 iteration can result in multiple Phase-2 tunnels, an option known as Perfect Forward Secrecy (PFS) can be used to ensure that each Phase-2 tunnel uses its own Phase-1 secure channel.

While IPsec VPNs can use manually configured encryption keys, it is preferable to use dynamically generated keys which are temporal in nature, meaning they change or “rotate” at certain intervals. A common way to dynamically key VPNs is through the IKE protocol.

IKE: Internet Key Exchange

Used to provide a mechanism for VPN gateways to exchange encryption keys dynamically, authenticate peer gateways, and negotiate proposals for encryption and authentication. Uses UDP port 500. IKE proposal exchange is “Phase-1” of the IPSec tunnel creation process. During IKE, the following is exchanged between VPN peers:

Phase-1: During Phase-1, the following is established during an IKE key exchange:

Encryption Algorithm
Hash Algorithm (Typically SHA-1 or MD5)
Authentication Method (Pre-shared keys, digital signatures, or public key encryption)
DH Group (Typically group 1, 2, or 5)
SA Lifetime (Measured in bytes or time)

Phase-1 modes:

Main Mode – When both VPN gateways use static IPs.
Aggressive Mode – When one VPN gateway uses DHCP.

Successful completion of Phase-1 results in a security association that contains the agreed upon protocols, authentication type, session keys, and SA lifetime.