[MidoNet-dev] Draft proposal: IPsec in midonet

Guillermo Ontañón guillermo at midokura.jp
Tue Feb 12 17:41:10 UTC 2013

Hello devs,

I'm including below my initial discussion about adding IPsec
support for midonet. It's still a bit rough but enough to
spark discussion. Some details (implementation related mostly)
are still blurry, I'm going to discuss the BGP part with Abel
tomorrow and create a test setup connecting to Amazon to get
those details right.

Feature description

IPSec VPN setup compatible with AWS's VPC private gateway.

The setup should include redundancy/failover.

Amazon VPC connectivity (See [0])

Since want fail-over, we need Amazon's 'dynamically-routed' flavour of
IPsec. This is what a fully redundant setup looks like:

                        Amazon VPC Subnets
                           ╎  GW-1   ╎
       link-local-address-A  ╎     ╎ link-local-address-B
             BGP-instance-A  ╎     ╎ BGP-instance-B
                             ╎     ╎
                         ┌───┘     └────┐
                         ╎              ╎
                    ┌────┴────┐    ┌────┴────┐
                    ╎ IPsec-A ╎    ╎ IPsec-B ╎
                    └────┬────┘    └────┬────┘
        Public-Address-A ╎              ╎ Public-Address-B
                         ╎              ╎
                         ╎              ╎
        Public-Address-C ╎              ╎ Public-Address-D
                    ┌────┴────┐    ┌────┴────┐
                    ╎ IPsec-C ╎    ╎ IPsec-D ╎
                    └────┬────┘    └────┬────┘
                         ╎              ╎
                         └───┐     ┌────┘
                             ╎     ╎
             BGP-instance-C  ╎     ╎ BGP-instance-D
       link-local-address-C  ╎     ╎ link-local-address-D
                           ╎  GW-2   ╎
                        Other cloud subnets

It works like this:

  - Two IPsec connections are established: IPsec-A - IPsec C, and
    IPsec B - IPsec D respectively.
  - For each IPsec connection pair (A-C, B-D) /30 network of link-local
    addresses is reserved. These addresses (LLA-A,B,C and D) are locally
    bound by each gateway behind the IPsec tunnels.
  - At this point, with the tunnels established, GW-1's traffic
    addressed to LLA-C will always take source address LLA-A and go
    through the A-C tunnel. Both gateways can now run two BGP instances
    each on the link-local-addresses, and packets between the subnets
    behind each gateway can start to flow through either IPsec tunnel.

(implementation note below - skip to next section if not interested)

Note that on this setup IPsec instances need to accept and encrypt
traffic from/to any network on either side, and they do not know that
when they establish the tunnel. Because the Linux kernel ipsec
implementation is policy based (you need Security Associations between
each pair of subnets that may communicate through the tunnel) this poses
a problem. It may be solved in two different ways (needs testing):

  - Statically add SA's localnet- Meaning that anytime the
    IPsec instance sees a packet coming from one of the known local
    networks the tuple of that network with will match and let
    the traffic through. This may be just fine, as we will be fully in
    control of what traffic makes it to the IPsec instance.
  - Have the BGP instance notify the IPsec instance to add the SA's
    dynamically. This is obviously more complicated, specially because
    we are not talking about MM state but linux kernel state inside of a

Digression: about the CloudStack implementation

See [1] for reference. CloudStack supports two types of crnnections:
L2TP and site-to-site. L2TP is meant for single users, not gateways, and
it tunnels l2tp packets make the user's machine be effectively in the
chosen VPC L2 network segment. This mode does not concern us for now.
Site-to-site VPNs in CloudStack appear to be old-fashioned policy based
IPsec tunnels. The full list of remote subnets is needed to configure
the connection and there's no mention of BGP.

However, CloudStack's site-to-site tunnels should still be compatible
with Amazon, ...without failover. Amazon added support for statically-
routed VPNs in September 2012 [2]. Seeing that Amazon extended their
offering with this feature and that it's the only mode that CloudStack
supports, it's probably worth considering supporting this mode too in

Proposal for adding dynamically-routed IPsec to Midonet

 - Configuration parameters. (REST API operations to be derived from
   these). To setup a redundant IPsec VPN you need to provide:

    - Two public IPs
    - Two addresses in different /30 link-local subnets, for BGP.
    - A virtual router. BGP will modify this router's routing table
      and both IPsec gateways will be connected directly to this router.
      User-visible parameters related to this connection to the virtual
      router should be kept to a minimum. It's not yet clear how the
      ipsec instances and bgp will be plugged to the vrouter.
    - Public IPs for the gateways on the other side.
    - Local network ranges to advertise to the other side via BGP.
    - IKE pre-shared key.
    - IPsec connection parameters: PFS mode, hash function, encryption
      function. Amazon is very strict on this, requiring exactly:
      Group-2, SHA-1, AES-128. IMHO that's a correct philosophy,
      sticking to the common minimum. If midonet is as strict as Amazon
      the user doesn't need to supply this.

 - This a sketch of what new REST API DTOs there would be:
    - IpsecPairingDto. With fields:
        - publicIp
        - peerIp
        - dhGroup (read-only if only 2 is allowed)
        - hashAlg (read-only if only sha-1 is allowed)
        - cryptoAlg (read-only if only aes128 is allowed)
    - BgpIpsecPairingDto. Fields:
        - localAsn
        - remoteAsn
        - localIp (must be link-local)
        - peerIp (must be in the same /30 network as localIp, it could
          be inferred from it)
    - DynamicVpnDto
        - virtualRouter
        - ipsecPairingA
        - ipsecPairingB
        - bgpPairingA
        - bgpPairingB
        - localSubnets
    - StaticVpnDto (when/if midonet supports this type of connection)
        - ipsecPairing
        - localSubnets
        - remoteSubnets

 - The virtual topology would look like this to the user:

                                          ╎ Public IP (interior vport)

                            ╵       ╎ IPsec VM ╎ X 2
                            ╵       ╎ instance ╎
                            ╵       └─────┬────┘
                       ┌────┴───┐         ╎
   BGP injected ──────>╎ Tenant ╎         ╎
       routes          ╎ Router ├─────────┘
                   ┌──────┘   └─────┐
             ┌─────┴────┐     ┌─────┴────┐
             ╎ Bridge-A ╎ ... ╎ Bridge-N ╎
             └──────────┘     └──────────┘




-- Guillermo Ontañón
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.midonet.org/pipermail/midonet-dev/attachments/20130212/c0bac7c1/attachment-0001.html>

More information about the MidoNet-dev mailing list