[MidoNet-dev] Draft Proposal: rate-limiting in Midonet

Guillermo Ontañón guillermo at midokura.jp
Tue Feb 19 18:00:07 UTC 2013

Hi devs,

Below is my draft spec for Midonet's rate-limiting feature, looking forward
to discussing it:

Feature Description

Add a rate-limiting feature targetted at cloud service providers, who
may want to limit the bandwidth available to their tenants in various

Use cases

There are three use cases, with varying degrees of importance/
usefulness, that Midonet could support:

Use case 1: Limit the uplink egress bandwitdh

Two reasons why this may make sense for a cloud service provider:

* Pricing options based on uplink speed.
* Resource sharing. The uplink is a shared resource and likely
  oversubscribed, rate limiting provides a limit into how much a given
  subscriber can affect resource allocation to the rest of subscribers.

Use case 2: Limit east-west traffic

Though much less important than use case #1. The motivations are

Use case 3. Limit the global egress bandwidth for a VM

Reasons for doing this:

* Having a global bandwidth limit for the VM makes it less necessary to
  explicitly support the 'east-west' use case. If that's the case,
  east-west bandwidth for a VM would be the global limit minus its
  current uplink usage.
* Fair sharing of the network link in a host among all guests. Note
  that, this is not real or perfect 'sharing' if the limits of the local
  VMs aren't somehow balanced to fully utilize the host's link.
* Limiting the guest's bandwidth to ensure services running on the host
  have guaranteed bandwidth.

Design/implementation implications of use cases

Use cases #1 and #2 clearly require that traffic be mapped to a rate
limiting queue on a per-flow basis. With the difference that use case #1
could have simpler design/implementations because all the rate-limited
traffic goes through a few edge nodes. We will call this flow-based

Use case #3 is very clearly the simplest from all points of view,
blindly rate limiting all traffic that ingresses the virtual topology
from a given VM vport. We will call this port-based rate-limiting.

Use cases #1 and #2 also pose the question of whether the rate-limiting
is applied to each VM (or rather, vport) independently or it can be
applied to each tenant or arbitrary group of VMs as a whole.

The latter, together with flow-based limiting, is a full generalisation
of the problem because it requires a distributed rate-limiting queue (on
top of the per-flow classification).  This is, for the time being, out
of the scope for this feature.

Furthermore, in a vport-local scenario, rate-limiting may be applied on
the ingress vport, on egress, or both.

Features supported in the Quantum NVP QoS feature

The NVP QoS feature for Quantum [0] is in code-review [1] state. It

  * Putting a rate-limiting queue in a VMs vport (no flow matching, not
    a distributed queue).
  * The settings for port's queue may be chosen individually or inherit
    from the queue set for the 'network'. So an admin may just configure
    one queue and have a whole set of VMs use those settings.
  * A rate-limiting queue can set:
    * max bandwidth rate
    * min bandwidth rate ???
    * qos_marking (unclear what it does, could it mark the packet for
      the kernel to custom further qos/processing on the packet?)
    * dscp (unclear too)

Proposed Rate Limiting feature for Midonet Diyari 13.06

Midonet will support ingress vport-local flow-based rate-limiting.
Meaning that:

  * Queues are local to a vport and applied on ingress only, a flow may
    only traverse one queue.
  * Flows are assigned to queues independently.

Therefore, all of the use cases discussed above would be covered,
although in a non-distributed manner.

Flow classification will use the existing chains/rules infrastructure.
Only one extension is needed: a new QOS action for rules. Upon matching
on a rule with a QOS action, Chain.apply() will:

  * Is the flow already marked for QOS by an eearlier rule?
    - Yes -> do nothing.
    - No  -> mark the flow with the queue id given by the rule
  * Continue processing on the next rule (this is a non-terminating

QOS rules will have a queueId UUID attribute. Queue objects need not
exist per-vport, their configuration will be used to create a queue
when traffic ingressing a vport is marked with a given queue id, but a
queue object is not bound to a vport it can be reused by rules
regardless of ingress port.

Necessary REST API changes

Changes in RuleDto:

  * 'flowAction' will get a new allowed value: 'queue'.
  * New field: queueId. It MUST have a value if flowAction is 'queue',
    and that value must be id of a valid QueueDto object.

A new QueueDto is created, with fields:
  * id
  * maxRateKbps Maximum ingress (into the virtual topology - egress from
    the pov of the VM) bandwidth.
  * qosMark (to-be-discussed) - this would mark the packet for matching
    in processing by the linux qos stack, letting admins setup more
    sophisticated QoS behaviour. This would use OVS skb-mark action.

A new top-level URL for the QueueDto:

  * /queues/ - ApplicationDto would get a new getQueues() method.
  * /queues/:queue_id/ (URL for a Queue - POST, DELETE, GET)


-- Guillermo Ontañón
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.midonet.org/pipermail/midonet-dev/attachments/20130219/78ae47b5/attachment.html>

More information about the MidoNet-dev mailing list