[MidoNet-dev] L4-7 Service Platform document on Wiki

Dan Mihai Dumitriu dan at midokura.com
Tue Feb 12 06:32:34 UTC 2013

Hi Pino,

On Tue, Feb 12, 2013 at 12:19 AM, Pino de Candia <gdecandia at midokura.com>wrote:

> Hi Dan, Stephen,
> On Monday, February 11, 2013 at 1:42 PM, Dan Mihai Dumitriu wrote:
> After a couple of brainstorming chats, I believe we have a strawman
> proposal for how to incorporate flow redirection into our current model,
> using the chain & rule framework, so that this is an incremental addition
> to our model. (kaizen! ;-)
> We introduce a new type of rule target which sends the flow to a port and
> interrupts 'normal' processing, where normal depends on the type of virtual
> device, i.e. routing for a vrouter and learning switching for a vswitch.
> I'd like to hear more about this vport. Is it an exterior port? In which
> case we're forwarding the flow out of the virtual topology to a single
> appliance? We brain-stormed about offering a layer of load-balancing here
> so that the inserted appliance could be scaled out. One way in which we can
> do that is that the new rule type allows a number of exterior vports to
> choose from. But I'd like to explore this a little further.
>  (This just came to mind, but note that in the case of a vswitch, ports
> attached to services should be marked as non-flooding, so that they don't
> normally participate in the learning switch.) Flows coming back into the
> virtual topology from a service would enter via another vport. (Pino, after
> talking with Stephen some more, we thought it might be too complex to
> reenter the virtual topology in the same chain, and with the same original
> ingress vport.)
> Sounds good to me.
> Another thing which comes to mind, however, is that if we want to do
> service chaining, this model is not sufficient, because each time the flow
> enters the virtual topology, from a service, it has lost the context of the
> original flow.  I have one extremely half-baked idea that could help here.
>  If we had extra space in the tunnel id, e.g. if we used GRE with 64 bit
> keys, or STT, could we use 32 of those bits to identify the original flow?
>  That way, even if a flow passed through a service and was modified, we can
> still identify it when it comes back into MN, and thus can enforce some
> order of services.  (I'm still thinking of how this might work; suggestions
> are welcome.)
> Can one of you guys give a concrete example of service chaining? I think I
> understand it in the abstract, but how long do those chains tend to be in
> practice? Is the point of tracking the whole chain (in the UUID) that you
> want to do decision caching across the whole chain? and hence collapse it
> to a single flow forwarding rule in the vswitch?

I can't give a good example of a use case for service chaining.  Stephen?

As far as this half-baked idea goes, I wasn't [yet] thinking of how to do
the decision caching, only the flow redirection.  I would not collapse the
decision into a single flow, as that might cover only a small set of use
cases, and it's an extreme optimization anyway.

The point of using the tunnel key to indicate the original flow is so that
we can always identify the original flow in rule matching conditions even
after that flow has gone through a service which mangles it.  (Currently
that would be a brand new flow to MN.)  I figure that if we can continue to
identify the original flow, and match on its metadata, as well as on the
service that it just came out of, then we can enforce an order of services.
 See what I mean?

 For the elasticity aspect, how about we introduce a level of indirection
> at the vport level, something like a 'group' vport, which will dispatch to
> one of a set of vports, based on flow hash.
> Ah, I thought you wanted to do this in the rule itself - but a 'group'
> vport also works. A flow hash (stateless) is not sufficient if we don't
> want connections to drop when the target set changes. Also, do we do any
> health-checks? (The same questions arise if we implement elasticity in the
> new rule type.

The system should have a notion of health of these service targets.
 However, I'd leave the precise mechanism out of the discussion now.  (I
think Stephen suggested that each service instance would register with its
local MM agent, and that agent would monitor the instance health, using the
control connection, and update it in ZK accordingly.)

> What do y'all think?
> I'm glad we're moving forward on this. I think we'll probably get it wrong
> (or not-quite-right) on the first pass. Also, this seems like the easy
> part. I suggest that we talk about service templates next OR the decision
> caching mechanism (per flow API between MidoNet and the inserted service).

Yeah Pino, this is definitely the easy part.  Forgive me for being lazy on
the weekend. :)


> On Thu, Feb 7, 2013 at 11:19 PM, Dan Mihai Dumitriu <dan at midokura.com>wrote:
> Hi Stephen,
> This is a great writeup!  I believe that we must take MidoNet into such a
> direction in order to provide the services needed by IaaS cloud users.
>  Please find my [yet incomplete because I'm falling asleep] comments below.
> Cheers,
> Dan
>    - Goals
>    - AWS
>    - Cisco 1000v vPath
>       - flow redirection
>          - allows VEM to direct traffic toward the service via tunneling
>          - what is required?  match() -> encapsulate to target (host,
>          vport) ?
>       - decision caching
>          - service node made a decision (think firewall) on a particular
>          flow and then download the flow entry to a target VEM (normally the VEM who
>          forwards the traffic to the service node)
>          - I suppose that a service node could be looking deeper than L4,
>          but that the decision must match a flow using fields up to L4.
>          - how is a flow identified?  still only by L2-L4 fields?
>       - service chaining
>          - enforce a traffic flow evaluation order upon which a flow
>          should traverse
>          - I don't quite get this one.
>        - vCEF
>       - what is SDDC?
>       - does this vCEF method use tunneling?
>       - I didn't quite understand what is unique about vCEF, other than
>       normalizing the
>    - Quantum Service Insertion
>       - Routed Mode
>          - How are flows specified?  Matching on IP?
>       - Floating/In-path Mode
>          - in this mode, the service is 'transparent' to the traffic?
>       - Redirected/Out-of-path Mode
>    - MidoNet Elastic Network Services Framework
>       - Service Launching Model
>          - case B sounds ideal, but I can see the potential pitfalls of
>          this approach.  the implementation for each cloud stack could end up being
>          somewhat different.  That said, it's not impossible, and may not be that
>          intrusive.  For example, OpenStack has a pluggable 'scheduler' which
>          chooses where VMs are launched.  The service platform could include such a
>          plugin in order to control where service VMs are launched.  (CloudStack may
>          have something as well.)  Furthermore, libraries like jclouds can be used
>          to abstract the cloud API (EC2 or OS or CS).  This is all very superficial
>          commentary, however. :)
>       - Service registration and configuration translation
>          - this sounds like what i've been thinking about, and somewhat
>          similar to the way that CloudStack manages its router virtual machine,
>          except that in the CS case the CS agent calls into the VM, rather than vice
>          versa.
>       - Service Insertion Model
>          - Flow Redirection
>             - "This flow should be installed as a wildcard flow into the
>             affected virtual router"
>                - we currently don't have a way to insert such low level
>                flows into the virtual router.
>                - we do have rule chains which can do flexible matching,
>                and I suppose we could make some kind of actions that do a redirect,
>                bypassing the routing stage.
>              - Elastic Service Insertion
>             - "Since we use the same hash function and extract the same
>             fields to perform the calculation, different ingress node will still yield
>             the same result"
>                - Is that really true? Can this decision always be
>                deterministic?
>             - What's the distinction between ECMP and VLB in this case?
>           - Flow Entries Management
>          - b is something like Direct Server Return
>          http://www.slideshare.net/cwestin63/l3-dsr-lspepresentation20120119
>           ?
>          - I suppose that an L7 LB would be in case c?
>          - (I got lost on much of the rest of this section)
>       - Management Interface
>       - Configuration Management
>       - Dynamic States Management
>          - indeed, makes sense; we'll have to hack up haproxy or some
>          such thing.  I wonder if nginx makes this easier.
>       - Elasticity
>    - Example Application: MidoNet Elastic Load Balancing (MNELB)
>       - Operating Model
>          - VIP is better than DNS name! :)
>       - Launch Model
>          - image maintenance
>             - the image may need to be tweaked for each stack (OS vs CS)
>           - Elasticity Management
>       - Traffic Flow
>          -
>       - HTTPS Flows
>          - SSL offloading and HTTPS with L7 load balancing
>          - Regarding the "session ID" in SSLv3, in this case where is the
>          SSL engine running?
>             - Is this assuming that the target server would do the actual
>             SSL?
>           - Appendix 1: What if our ingress router cannot handle that
>    many flows?
>       - Why is the exact match unscalable?  Is that because the kmod flow
>       hashtable cannot scale?  How large do we think it can scale?  Is it just a
>       matter of memory use?
>       - Does TCAM really solve our problem?  It seems that in order to
>       'virtualize' the network, we really do need to run a simulation per flow.
>        That's because we are using information that is not available in the tcam
>       to make the decision about what to do with the flow.
>          - We would not be able to do stateful flow processing in the
>          ingress edge.
>          - Rather, probably what we'd do is forward the flow to a
>          'service node' which would run the simulation, and then push back the flow
>          decision to the ingress edge, potentially bypassing the service node,
>          assuming only stateless L2-L4 processing needed to be done.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.midonet.org/pipermail/midonet-dev/attachments/20130212/af008bc7/attachment.html>

More information about the MidoNet-dev mailing list