[MidoNet-dev] Draft Proposal: rate-limiting in Midonet
guillermo at midokura.jp
Fri Feb 22 11:48:31 UTC 2013
On Thu, Feb 21, 2013 at 9:01 AM, Dan Mihai Dumitriu <dan at midokura.com>wrote:
> This looks good to Me Guillermo!
> On Wed, Feb 20, 2013 at 3:00 AM, Guillermo Ontañón <guillermo at midokura.jp>wrote:
>> Hi devs,
>> Below is my draft spec for Midonet's rate-limiting feature, looking
>> forward to discussing it:
>> Feature Description
>> Add a rate-limiting feature targetted at cloud service providers, who
>> may want to limit the bandwidth available to their tenants in various
>> Use cases
>> There are three use cases, with varying degrees of importance/
>> usefulness, that Midonet could support:
>> Use case 1: Limit the uplink egress bandwitdh
>> Two reasons why this may make sense for a cloud service provider:
>> * Pricing options based on uplink speed.
>> * Resource sharing. The uplink is a shared resource and likely
>> oversubscribed, rate limiting provides a limit into how much a given
>> subscriber can affect resource allocation to the rest of subscribers.
>> Use case 2: Limit east-west traffic
>> Though much less important than use case #1. The motivations are
>> Use case 3. Limit the global egress bandwidth for a VM
>> Reasons for doing this:
>> * Having a global bandwidth limit for the VM makes it less necessary to
>> explicitly support the 'east-west' use case. If that's the case,
>> east-west bandwidth for a VM would be the global limit minus its
>> current uplink usage.
>> * Fair sharing of the network link in a host among all guests. Note
>> that, this is not real or perfect 'sharing' if the limits of the local
>> VMs aren't somehow balanced to fully utilize the host's link.
>> * Limiting the guest's bandwidth to ensure services running on the host
>> have guaranteed bandwidth.
>> Design/implementation implications of use cases
>> Use cases #1 and #2 clearly require that traffic be mapped to a rate
>> limiting queue on a per-flow basis. With the difference that use case #1
>> could have simpler design/implementations because all the rate-limited
>> traffic goes through a few edge nodes. We will call this flow-based
>> Use case #3 is very clearly the simplest from all points of view,
>> blindly rate limiting all traffic that ingresses the virtual topology
>> from a given VM vport. We will call this port-based rate-limiting.
>> Use cases #1 and #2 also pose the question of whether the rate-limiting
>> is applied to each VM (or rather, vport) independently or it can be
>> applied to each tenant or arbitrary group of VMs as a whole.
>> The latter, together with flow-based limiting, is a full generalisation
>> of the problem because it requires a distributed rate-limiting queue (on
>> top of the per-flow classification). This is, for the time being, out
>> of the scope for this feature.
>> Furthermore, in a vport-local scenario, rate-limiting may be applied on
>> the ingress vport, on egress, or both.
>> Features supported in the Quantum NVP QoS feature
>> The NVP QoS feature for Quantum  is in code-review  state. It
>> * Putting a rate-limiting queue in a VMs vport (no flow matching, not
>> a distributed queue).
>> * The settings for port's queue may be chosen individually or inherit
>> from the queue set for the 'network'. So an admin may just configure
>> one queue and have a whole set of VMs use those settings.
>> * A rate-limiting queue can set:
>> * max bandwidth rate
>> * min bandwidth rate ???
>> * qos_marking (unclear what it does, could it mark the packet for
>> the kernel to custom further qos/processing on the packet?)
> Not sure what this is. It might be setting the mark field on the skb,
> which could then be matched by linux tc.
Yep, that's what I understand, but I want to make 100% sure.
>> * dscp (unclear too)
> Sets the DSCP field of the outer ip header. This could be matched by the
> underlay switches and mapped to a traffic class, which needs to have been
> setup a priori. This is useful as well. (I might put this into a
> different action though.)
So we'd have three new non-terminating actions: QUEUE, QOS_MARK, DSCP.
If we have separate actions, maybe the Queue object is not needed until
when/if we do more fancy qos, for now it would only have a 'max-bandwidth'
field and that could be set with the QOS action, or rather RATE_LIMIT.
>> Proposed Rate Limiting feature for Midonet Diyari 13.06
>> Midonet will support ingress vport-local flow-based rate-limiting.
>> Meaning that:
>> * Queues are local to a vport and applied on ingress only, a flow may
>> only traverse one queue.
>> * Flows are assigned to queues independently.
>> Therefore, all of the use cases discussed above would be covered,
>> although in a non-distributed manner.
>> Flow classification will use the existing chains/rules infrastructure.
>> Only one extension is needed: a new QOS action for rules. Upon matching
>> on a rule with a QOS action, Chain.apply() will:
>> * Is the flow already marked for QOS by an eearlier rule?
>> - Yes -> do nothing.
>> - No -> mark the flow with the queue id given by the rule
>> * Continue processing on the next rule (this is a non-terminating
>> QOS rules will have a queueId UUID attribute. Queue objects need not
>> exist per-vport, their configuration will be used to create a queue
>> when traffic ingressing a vport is marked with a given queue id, but a
>> queue object is not bound to a vport it can be reused by rules
>> regardless of ingress port.
>> Necessary REST API changes
>> Changes in RuleDto:
>> * 'flowAction' will get a new allowed value: 'queue'.
>> * New field: queueId. It MUST have a value if flowAction is 'queue',
>> and that value must be id of a valid QueueDto object.
>> A new QueueDto is created, with fields:
>> * id
>> * maxRateKbps Maximum ingress (into the virtual topology - egress from
>> the pov of the VM) bandwidth.
>> * qosMark (to-be-discussed) - this would mark the packet for matching
>> in processing by the linux qos stack, letting admins setup more
>> sophisticated QoS behaviour. This would use OVS skb-mark action.
>> A new top-level URL for the QueueDto:
>> * /queues/ - ApplicationDto would get a new getQueues() method.
>> * /queues/:queue_id/ (URL for a Queue - POST, DELETE, GET)
>> -- Guillermo Ontañón
>> MidoNet-dev mailing list
>> MidoNet-dev at lists.midonet.org
-- Guillermo Ontañón
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the MidoNet-dev