[MidoNet-dev] Draft Proposal: rate-limiting in Midonet

Guillermo Ontañón guillermo at midokura.jp
Tue Feb 19 18:00:07 UTC 2013


Hi devs,

Below is my draft spec for Midonet's rate-limiting feature, looking forward
to discussing it:

Feature Description
-------------------

Add a rate-limiting feature targetted at cloud service providers, who
may want to limit the bandwidth available to their tenants in various
ways.

Use cases
---------

There are three use cases, with varying degrees of importance/
usefulness, that Midonet could support:

Use case 1: Limit the uplink egress bandwitdh

Two reasons why this may make sense for a cloud service provider:

* Pricing options based on uplink speed.
* Resource sharing. The uplink is a shared resource and likely
  oversubscribed, rate limiting provides a limit into how much a given
  subscriber can affect resource allocation to the rest of subscribers.

Use case 2: Limit east-west traffic

Though much less important than use case #1. The motivations are
similar.

Use case 3. Limit the global egress bandwidth for a VM

Reasons for doing this:

* Having a global bandwidth limit for the VM makes it less necessary to
  explicitly support the 'east-west' use case. If that's the case,
  east-west bandwidth for a VM would be the global limit minus its
  current uplink usage.
* Fair sharing of the network link in a host among all guests. Note
  that, this is not real or perfect 'sharing' if the limits of the local
  VMs aren't somehow balanced to fully utilize the host's link.
* Limiting the guest's bandwidth to ensure services running on the host
  have guaranteed bandwidth.

Design/implementation implications of use cases
-----------------------------------------------

Use cases #1 and #2 clearly require that traffic be mapped to a rate
limiting queue on a per-flow basis. With the difference that use case #1
could have simpler design/implementations because all the rate-limited
traffic goes through a few edge nodes. We will call this flow-based
rate-limiting.

Use case #3 is very clearly the simplest from all points of view,
blindly rate limiting all traffic that ingresses the virtual topology
from a given VM vport. We will call this port-based rate-limiting.

Use cases #1 and #2 also pose the question of whether the rate-limiting
is applied to each VM (or rather, vport) independently or it can be
applied to each tenant or arbitrary group of VMs as a whole.

The latter, together with flow-based limiting, is a full generalisation
of the problem because it requires a distributed rate-limiting queue (on
top of the per-flow classification).  This is, for the time being, out
of the scope for this feature.

Furthermore, in a vport-local scenario, rate-limiting may be applied on
the ingress vport, on egress, or both.

Features supported in the Quantum NVP QoS feature
-------------------------------------------------

The NVP QoS feature for Quantum [0] is in code-review [1] state. It
offers:

  * Putting a rate-limiting queue in a VMs vport (no flow matching, not
    a distributed queue).
  * The settings for port's queue may be chosen individually or inherit
    from the queue set for the 'network'. So an admin may just configure
    one queue and have a whole set of VMs use those settings.
  * A rate-limiting queue can set:
    * max bandwidth rate
    * min bandwidth rate ???
    * qos_marking (unclear what it does, could it mark the packet for
      the kernel to custom further qos/processing on the packet?)
    * dscp (unclear too)

Proposed Rate Limiting feature for Midonet Diyari 13.06
-------------------------------------------------------

Midonet will support ingress vport-local flow-based rate-limiting.
Meaning that:

  * Queues are local to a vport and applied on ingress only, a flow may
    only traverse one queue.
  * Flows are assigned to queues independently.

Therefore, all of the use cases discussed above would be covered,
although in a non-distributed manner.

Flow classification will use the existing chains/rules infrastructure.
Only one extension is needed: a new QOS action for rules. Upon matching
on a rule with a QOS action, Chain.apply() will:

  * Is the flow already marked for QOS by an eearlier rule?
    - Yes -> do nothing.
    - No  -> mark the flow with the queue id given by the rule
  * Continue processing on the next rule (this is a non-terminating
    rule)

QOS rules will have a queueId UUID attribute. Queue objects need not
exist per-vport, their configuration will be used to create a queue
when traffic ingressing a vport is marked with a given queue id, but a
queue object is not bound to a vport it can be reused by rules
regardless of ingress port.

Necessary REST API changes
--------------------------

Changes in RuleDto:

  * 'flowAction' will get a new allowed value: 'queue'.
  * New field: queueId. It MUST have a value if flowAction is 'queue',
    and that value must be id of a valid QueueDto object.

A new QueueDto is created, with fields:
  * id
  * maxRateKbps Maximum ingress (into the virtual topology - egress from
    the pov of the VM) bandwidth.
  * qosMark (to-be-discussed) - this would mark the packet for matching
    in processing by the linux qos stack, letting admins setup more
    sophisticated QoS behaviour. This would use OVS skb-mark action.

A new top-level URL for the QueueDto:

  * /queues/ - ApplicationDto would get a new getQueues() method.
  * /queues/:queue_id/ (URL for a Queue - POST, DELETE, GET)



Regards,

-- Guillermo Ontañón
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.midonet.org/pipermail/midonet-dev/attachments/20130219/78ae47b5/attachment.html>


More information about the MidoNet-dev mailing list