[MidoNet-dev] Draft proposal: IPsec in midonet

Guillermo Ontañón guillermo at midokura.jp
Wed Feb 13 11:22:18 UTC 2013


Hi Dan,

Good questions, my comments below...


On Wed, Feb 13, 2013 at 6:43 AM, Dan Mihai Dumitriu <dan at midokura.com>wrote:

> Hi Guillermo,
>
> I was just about to send a message with a similar proposal, and you just
> beat me to it! :) This looks great.
>
> There are a couple of points I want to raise.
>
> - since one of our objectives is to connect a private cloud to an AWS VPC,
> is it correct that our IPSec VPN will have to run in client mode as well?
>  Actually, I don't know how this stuff is supposed to work.  Is one side a
> server and the other a client?  Or is the configuration of such a VPN
> symmetric?
>

The configuration is symmetric, there's no server and client. Once set up,
both IPsec gateways should try to contact the other (UDP port 500).


> - for managing the IPSec instances, do we want to leverage the VMs of the
> cloud stack on top of which we're running?  We talked about running them in
> containers before.  I think I might be coming back around to thinking that
> VMs are the way to go. :) If the performance is good enough, something
> which has to be measured.  (CloudStack runs all services in VMs.)
>

That's the biggest question we have, I was just discussing this with Abel.
I don't have a final opinion yet, before I do want to do a real test with
Amazon. Some thoughts:

  * Containers will not be possible if the IPsec SPDB is not per-container
too. I haven't tried but have been unable to find anyone who has gotten it
to work.
  * The downside of using a VM is maintaining the image with updates, etc.
  * A second downside of the VM is that we'd be redoing some of the work in
the current BGP feature.
  * The upside of using a VM is very attractive to me: you have this black
box with two ports that you can plug anywhere into the virtual topology. In
this case we'd put BGP and IPsec in the VM, plugged to a tenant router on
one side and to the provider router (or even a second port of the tenant
router) on the other.
  * Unless we do a clever trick (connection through the datapath?) BGP and
IPsec have to run in the same container/vm/whatever. Because when BGP sends
packets out of its link-local-address the destination address is on the
same ip subnet and on other side of the tunnel, so they have to be fed into
the networking stack that has the IPsec SPDB. I mention this because Abel
and I have been discussing ways of making the BGP and IPsec instances
independent, reusing our current BGP setup.


> - CloudStack already has an IPSec VPN in the router VM.  We just spoke
> yesterday about leveraging that to support the feature when running
> together with MN.  In order to add a BGP option, we have to extend the CS
> API itself.  To make this fault tolerant, we should probably prepare
> another image.
>
> - On the OpenStack side, in order to consider this a complete feature, we
> also need to add APIs to quantum.  Does anyone know if there is any ongoing
> blueprint for a VPC feature?  In addition, we need to prepare a VM with the
> ipsec code and some agent that we can control from MN.  For the control
> connection between the MN agent and the agent in the VM, one way to go is
> to do as CloudStack does, that is to create another interface that is
> bridged to the host with a link local address.
>

Sounds like we could create a VM base image with an extensible agent and
VM-Midonet port that 3rd parties could build on.


>
> Thoughts?
>
> Cheers,
> Dan
>
>
> On Wed, Feb 13, 2013 at 2:41 AM, Guillermo Ontañón <guillermo at midokura.jp>wrote:
>
>> Hello devs,
>>
>> I'm including below my initial discussion about adding IPsec
>> support for midonet. It's still a bit rough but enough to
>> spark discussion. Some details (implementation related mostly)
>> are still blurry, I'm going to discuss the BGP part with Abel
>> tomorrow and create a test setup connecting to Amazon to get
>> those details right.
>>
>> Feature description
>> -------------------
>>
>> IPSec VPN setup compatible with AWS's VPC private gateway.
>>
>> The setup should include redundancy/failover.
>>
>> Amazon VPC connectivity (See [0])
>> ---------------------------------
>>
>> Since want fail-over, we need Amazon's 'dynamically-routed' flavour of
>> IPsec. This is what a fully redundant setup looks like:
>>
>>                         ~~~~~~~~~~~~~~~~~~
>>                         Amazon VPC Subnets
>>                         ~~~~~~~~┬~~~~~~~~~
>>>>                            ┌────┴────┐
>>                            ╎  GW-1   ╎
>>                            └─┬─────┬─┘
>>        link-local-address-A  ╎     ╎ link-local-address-B
>>              BGP-instance-A  ╎     ╎ BGP-instance-B
>>                              ╎     ╎
>>                          ┌───┘     └────┐
>>                          ╎              ╎
>>                     ┌────┴────┐    ┌────┴────┐
>>                     ╎ IPsec-A ╎    ╎ IPsec-B ╎
>>                     └────┬────┘    └────┬────┘
>>         Public-Address-A ╎              ╎ Public-Address-B
>>                          ╎              ╎
>>                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>                              Internet
>>                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>                          ╎              ╎
>>         Public-Address-C ╎              ╎ Public-Address-D
>>                     ┌────┴────┐    ┌────┴────┐
>>                     ╎ IPsec-C ╎    ╎ IPsec-D ╎
>>                     └────┬────┘    └────┬────┘
>>                          ╎              ╎
>>                          └───┐     ┌────┘
>>                              ╎     ╎
>>              BGP-instance-C  ╎     ╎ BGP-instance-D
>>        link-local-address-C  ╎     ╎ link-local-address-D
>>                            ┌─┴─────┴─┐
>>                            ╎  GW-2   ╎
>>                            └────┬────┘
>>>>                         ~~~~~~~~┴~~~~~~~~~~
>>                         Other cloud subnets
>>                         ~~~~~~~~~~~~~~~~~~~
>>
>> It works like this:
>>
>>   - Two IPsec connections are established: IPsec-A - IPsec C, and
>>     IPsec B - IPsec D respectively.
>>   - For each IPsec connection pair (A-C, B-D) /30 network of link-local
>>     addresses is reserved. These addresses (LLA-A,B,C and D) are locally
>>     bound by each gateway behind the IPsec tunnels.
>>   - At this point, with the tunnels established, GW-1's traffic
>>     addressed to LLA-C will always take source address LLA-A and go
>>     through the A-C tunnel. Both gateways can now run two BGP instances
>>     each on the link-local-addresses, and packets between the subnets
>>     behind each gateway can start to flow through either IPsec tunnel.
>>
>> (implementation note below - skip to next section if not interested)
>>
>> Note that on this setup IPsec instances need to accept and encrypt
>> traffic from/to any network on either side, and they do not know that
>> when they establish the tunnel. Because the Linux kernel ipsec
>> implementation is policy based (you need Security Associations between
>> each pair of subnets that may communicate through the tunnel) this poses
>> a problem. It may be solved in two different ways (needs testing):
>>
>>   - Statically add SA's localnet-0.0.0.0/0. Meaning that anytime the
>>     IPsec instance sees a packet coming from one of the known local
>>     networks the tuple of that network with 0.0.0.0/0 will match and let
>>     the traffic through. This may be just fine, as we will be fully in
>>     control of what traffic makes it to the IPsec instance.
>>   - Have the BGP instance notify the IPsec instance to add the SA's
>>     dynamically. This is obviously more complicated, specially because
>>     we are not talking about MM state but linux kernel state inside of a
>>     VM.
>>
>> Digression: about the CloudStack implementation
>> -----------------------------------------------
>>
>> See [1] for reference. CloudStack supports two types of crnnections:
>> L2TP and site-to-site. L2TP is meant for single users, not gateways, and
>> it tunnels l2tp packets make the user's machine be effectively in the
>> chosen VPC L2 network segment. This mode does not concern us for now.
>> Site-to-site VPNs in CloudStack appear to be old-fashioned policy based
>> IPsec tunnels. The full list of remote subnets is needed to configure
>> the connection and there's no mention of BGP.
>>
>> However, CloudStack's site-to-site tunnels should still be compatible
>> with Amazon, ...without failover. Amazon added support for statically-
>> routed VPNs in September 2012 [2]. Seeing that Amazon extended their
>> offering with this feature and that it's the only mode that CloudStack
>> supports, it's probably worth considering supporting this mode too in
>> Midonet.
>>
>> Proposal for adding dynamically-routed IPsec to Midonet
>> -------------------------------------------------------
>>
>>  - Configuration parameters. (REST API operations to be derived from
>>    these). To setup a redundant IPsec VPN you need to provide:
>>
>>     - Two public IPs
>>     - Two addresses in different /30 link-local subnets, for BGP.
>>     - A virtual router. BGP will modify this router's routing table
>>       and both IPsec gateways will be connected directly to this router.
>>       User-visible parameters related to this connection to the virtual
>>       router should be kept to a minimum. It's not yet clear how the
>>       ipsec instances and bgp will be plugged to the vrouter.
>>     - Public IPs for the gateways on the other side.
>>     - Local network ranges to advertise to the other side via BGP.
>>     - IKE pre-shared key.
>>     - IPsec connection parameters: PFS mode, hash function, encryption
>>       function. Amazon is very strict on this, requiring exactly:
>>       Group-2, SHA-1, AES-128. IMHO that's a correct philosophy,
>>       sticking to the common minimum. If midonet is as strict as Amazon
>>       the user doesn't need to supply this.
>>
>>  - This a sketch of what new REST API DTOs there would be:
>>     - IpsecPairingDto. With fields:
>>         - publicIp
>>         - peerIp
>>         - dhGroup (read-only if only 2 is allowed)
>>         - hashAlg (read-only if only sha-1 is allowed)
>>         - cryptoAlg (read-only if only aes128 is allowed)
>>     - BgpIpsecPairingDto. Fields:
>>         - localAsn
>>         - remoteAsn
>>         - localIp (must be link-local)
>>         - peerIp (must be in the same /30 network as localIp, it could
>>           be inferred from it)
>>     - DynamicVpnDto
>>         - virtualRouter
>>         - ipsecPairingA
>>         - ipsecPairingB
>>         - bgpPairingA
>>         - bgpPairingB
>>         - localSubnets
>>     - StaticVpnDto (when/if midonet supports this type of connection)
>>         - ipsecPairing
>>         - localSubnets
>>         - remoteSubnets
>>
>>  - The virtual topology would look like this to the user:
>>
>>                                           ╎ Public IP (interior vport)
>>
>>                                     ┌─────┴────┐
>>                             ╵       ╎ IPsec VM ╎ X 2
>>                             ╵       ╎ instance ╎
>>                             ╵       └─────┬────┘
>>                        ┌────┴───┐         ╎
>>    BGP injected ──────>╎ Tenant ╎         ╎
>>        routes          ╎ Router ├─────────┘
>>                        └──┬───┬─┘
>>                    ┌──────┘   └─────┐
>>              ┌─────┴────┐     ┌─────┴────┐
>>              ╎ Bridge-A ╎ ... ╎ Bridge-N ╎
>>              └──────────┘     └──────────┘
>>
>>
>> Links
>> -----
>>
>> [0]
>> http://docs.aws.amazon.com/AmazonVPC/latest/NetworkAdminGuide/Introduction.html
>> [1]
>> http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Admin_Guide/vpn.html#site-to-site-vpn
>> [2]
>> http://aws.typepad.com/aws/2012/09/amazon-vpc-additional-vpn-features.html
>>
>>
>>
>> Regards,
>>
>> -- Guillermo Ontañón
>>
>
>


-- 
-- Guillermo Ontañón
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.midonet.org/pipermail/midonet-dev/attachments/20130213/8758fd54/attachment-0001.html>


More information about the MidoNet-dev mailing list