[MidoNet-dev] Draft Proposal: rate-limiting in Midonet

Dan Mihai Dumitriu dan at midokura.com
Thu Feb 21 08:01:30 UTC 2013

This looks good to Me Guillermo!

On Wed, Feb 20, 2013 at 3:00 AM, Guillermo Ontañón <guillermo at midokura.jp>wrote:

> Hi devs,
> Below is my draft spec for Midonet's rate-limiting feature, looking
> forward to discussing it:
> Feature Description
> -------------------
> Add a rate-limiting feature targetted at cloud service providers, who
> may want to limit the bandwidth available to their tenants in various
> ways.
> Use cases
> ---------
> There are three use cases, with varying degrees of importance/
> usefulness, that Midonet could support:
> Use case 1: Limit the uplink egress bandwitdh
> Two reasons why this may make sense for a cloud service provider:
> * Pricing options based on uplink speed.
> * Resource sharing. The uplink is a shared resource and likely
>   oversubscribed, rate limiting provides a limit into how much a given
>   subscriber can affect resource allocation to the rest of subscribers.
> Use case 2: Limit east-west traffic
> Though much less important than use case #1. The motivations are
> similar.
> Use case 3. Limit the global egress bandwidth for a VM
> Reasons for doing this:
> * Having a global bandwidth limit for the VM makes it less necessary to
>   explicitly support the 'east-west' use case. If that's the case,
>   east-west bandwidth for a VM would be the global limit minus its
>   current uplink usage.
> * Fair sharing of the network link in a host among all guests. Note
>   that, this is not real or perfect 'sharing' if the limits of the local
>   VMs aren't somehow balanced to fully utilize the host's link.
> * Limiting the guest's bandwidth to ensure services running on the host
>   have guaranteed bandwidth.
> Design/implementation implications of use cases
> -----------------------------------------------
> Use cases #1 and #2 clearly require that traffic be mapped to a rate
> limiting queue on a per-flow basis. With the difference that use case #1
> could have simpler design/implementations because all the rate-limited
> traffic goes through a few edge nodes. We will call this flow-based
> rate-limiting.
> Use case #3 is very clearly the simplest from all points of view,
> blindly rate limiting all traffic that ingresses the virtual topology
> from a given VM vport. We will call this port-based rate-limiting.
> Use cases #1 and #2 also pose the question of whether the rate-limiting
> is applied to each VM (or rather, vport) independently or it can be
> applied to each tenant or arbitrary group of VMs as a whole.
> The latter, together with flow-based limiting, is a full generalisation
> of the problem because it requires a distributed rate-limiting queue (on
> top of the per-flow classification).  This is, for the time being, out
> of the scope for this feature.
> Furthermore, in a vport-local scenario, rate-limiting may be applied on
> the ingress vport, on egress, or both.
> Features supported in the Quantum NVP QoS feature
> -------------------------------------------------
> The NVP QoS feature for Quantum [0] is in code-review [1] state. It
> offers:
>   * Putting a rate-limiting queue in a VMs vport (no flow matching, not
>     a distributed queue).
>   * The settings for port's queue may be chosen individually or inherit
>     from the queue set for the 'network'. So an admin may just configure
>     one queue and have a whole set of VMs use those settings.
>   * A rate-limiting queue can set:
>     * max bandwidth rate
>     * min bandwidth rate ???
>     * qos_marking (unclear what it does, could it mark the packet for
>       the kernel to custom further qos/processing on the packet?)

Not sure what this is.  It might be setting the mark field on the skb,
which could then be matched by linux tc.

>     * dscp (unclear too)

Sets the DSCP field of the outer ip header.  This could be matched by the
underlay switches and mapped to a traffic class, which needs to have been
setup a priori.  This is useful as well.  (I might put this into a
different action though.)

> Proposed Rate Limiting feature for Midonet Diyari 13.06
> -------------------------------------------------------
> Midonet will support ingress vport-local flow-based rate-limiting.
> Meaning that:
>   * Queues are local to a vport and applied on ingress only, a flow may
>     only traverse one queue.
>   * Flows are assigned to queues independently.
> Therefore, all of the use cases discussed above would be covered,
> although in a non-distributed manner.
> Flow classification will use the existing chains/rules infrastructure.
> Only one extension is needed: a new QOS action for rules. Upon matching
> on a rule with a QOS action, Chain.apply() will:
>   * Is the flow already marked for QOS by an eearlier rule?
>     - Yes -> do nothing.
>     - No  -> mark the flow with the queue id given by the rule
>   * Continue processing on the next rule (this is a non-terminating
>     rule)
> QOS rules will have a queueId UUID attribute. Queue objects need not
> exist per-vport, their configuration will be used to create a queue
> when traffic ingressing a vport is marked with a given queue id, but a
> queue object is not bound to a vport it can be reused by rules
> regardless of ingress port.
> Necessary REST API changes
> --------------------------
> Changes in RuleDto:
>   * 'flowAction' will get a new allowed value: 'queue'.
>   * New field: queueId. It MUST have a value if flowAction is 'queue',
>     and that value must be id of a valid QueueDto object.
> A new QueueDto is created, with fields:
>   * id
>   * maxRateKbps Maximum ingress (into the virtual topology - egress from
>     the pov of the VM) bandwidth.
>   * qosMark (to-be-discussed) - this would mark the packet for matching
>     in processing by the linux qos stack, letting admins setup more
>     sophisticated QoS behaviour. This would use OVS skb-mark action.
> A new top-level URL for the QueueDto:
>   * /queues/ - ApplicationDto would get a new getQueues() method.
>   * /queues/:queue_id/ (URL for a Queue - POST, DELETE, GET)
> Regards,
> -- Guillermo Ontañón
> _______________________________________________
> MidoNet-dev mailing list
> MidoNet-dev at lists.midonet.org
> http://lists.midonet.org/listinfo/midonet-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.midonet.org/pipermail/midonet-dev/attachments/20130221/ed147f99/attachment-0001.html>

More information about the MidoNet-dev mailing list