The QoS sample application demonstrates the use of the DPDK to provide QoS scheduling.
The architecture of the QoS scheduler application is shown in the following figure.
QoS Scheduler Application Architecture
There are two flavors of the runtime execution for this application, with two or three threads per each packet flow configuration being used. The RX thread reads packets from the RX port, classifies the packets based on the double VLAN (outer and inner) and the lower byte of the IP destination address and puts them into the ring queue. The worker thread dequeues the packets from the ring and calls the QoS scheduler enqueue/dequeue functions. If a separate TX core is used, these are sent to the TX ring. Otherwise, they are sent directly to the TX port. The TX thread, if present, reads from the TX ring and write the packets to the TX port.
To compile the sample application see Compiling the Sample Applications.
The application is located in the qos_sched sub-directory.
Note
This application is intended as a linux only.
Note
To get statistics on the sample app using the command line interface as described in the next section, DPDK must be compiled defining CONFIG_RTE_SCHED_COLLECT_STATS, which can be done by changing the configuration file for the specific target to be compiled.
Note
In order to run the application, a total of at least 4 G of huge pages must be set up for each of the used sockets (depending on the cores in use).
The application has a number of command line options:
./qos_sched [EAL options] -- <APP PARAMS>
Mandatory application parameters include:
Optional application parameters include:
Refer to DPDK Getting Started Guide for general information on running applications and the Environment Abstraction Layer (EAL) options.
The profile configuration file defines all the port/subport/pipe/traffic class/queue parameters needed for the QoS scheduler configuration.
The profile file has the following format:
; port configuration [port]
frame overhead = 24
number of subports per port = 1
; Subport configuration
[subport 0]
number of pipes per subport = 4096
queue sizes = 64 64 64 64 64 64 64 64 64 64 64 64 64
tb rate = 1250000000; Bytes per second
tb size = 1000000; Bytes
tc 0 rate = 1250000000; Bytes per second
tc 1 rate = 1250000000; Bytes per second
tc 2 rate = 1250000000; Bytes per second
tc 3 rate = 1250000000; Bytes per second
tc 4 rate = 1250000000; Bytes per second
tc 5 rate = 1250000000; Bytes per second
tc 6 rate = 1250000000; Bytes per second
tc 7 rate = 1250000000; Bytes per second
tc 8 rate = 1250000000; Bytes per second
tc 9 rate = 1250000000; Bytes per second
tc 10 rate = 1250000000; Bytes per second
tc 11 rate = 1250000000; Bytes per second
tc 12 rate = 1250000000; Bytes per second
tc period = 10; Milliseconds
tc oversubscription period = 10; Milliseconds
pipe 0-4095 = 0; These pipes are configured with pipe profile 0
; Pipe configuration
[pipe profile 0]
tb rate = 305175; Bytes per second
tb size = 1000000; Bytes
tc 0 rate = 305175; Bytes per second
tc 1 rate = 305175; Bytes per second
tc 2 rate = 305175; Bytes per second
tc 3 rate = 305175; Bytes per second
tc 4 rate = 305175; Bytes per second
tc 5 rate = 305175; Bytes per second
tc 6 rate = 305175; Bytes per second
tc 7 rate = 305175; Bytes per second
tc 8 rate = 305175; Bytes per second
tc 9 rate = 305175; Bytes per second
tc 10 rate = 305175; Bytes per second
tc 11 rate = 305175; Bytes per second
tc 12 rate = 305175; Bytes per second
tc period = 40; Milliseconds
tc 0 oversubscription weight = 1
tc 1 oversubscription weight = 1
tc 2 oversubscription weight = 1
tc 3 oversubscription weight = 1
tc 4 oversubscription weight = 1
tc 5 oversubscription weight = 1
tc 6 oversubscription weight = 1
tc 7 oversubscription weight = 1
tc 8 oversubscription weight = 1
tc 9 oversubscription weight = 1
tc 10 oversubscription weight = 1
tc 11 oversubscription weight = 1
tc 12 oversubscription weight = 1
tc 12 wrr weights = 1 1 1 1
; RED params per traffic class and color (Green / Yellow / Red)
[red]
tc 0 wred min = 48 40 32
tc 0 wred max = 64 64 64
tc 0 wred inv prob = 10 10 10
tc 0 wred weight = 9 9 9
tc 1 wred min = 48 40 32
tc 1 wred max = 64 64 64
tc 1 wred inv prob = 10 10 10
tc 1 wred weight = 9 9 9
tc 2 wred min = 48 40 32
tc 2 wred max = 64 64 64
tc 2 wred inv prob = 10 10 10
tc 2 wred weight = 9 9 9
tc 3 wred min = 48 40 32
tc 3 wred max = 64 64 64
tc 3 wred inv prob = 10 10 10
tc 3 wred weight = 9 9 9
tc 4 wred min = 48 40 32
tc 4 wred max = 64 64 64
tc 4 wred inv prob = 10 10 10
tc 4 wred weight = 9 9 9
tc 5 wred min = 48 40 32
tc 5 wred max = 64 64 64
tc 5 wred inv prob = 10 10 10
tc 5 wred weight = 9 9 9
tc 6 wred min = 48 40 32
tc 6 wred max = 64 64 64
tc 6 wred inv prob = 10 10 10
tc 6 wred weight = 9 9 9
tc 7 wred min = 48 40 32
tc 7 wred max = 64 64 64
tc 7 wred inv prob = 10 10 10
tc 7 wred weight = 9 9 9
tc 8 wred min = 48 40 32
tc 8 wred max = 64 64 64
tc 8 wred inv prob = 10 10 10
tc 8 wred weight = 9 9 9
tc 9 wred min = 48 40 32
tc 9 wred max = 64 64 64
tc 9 wred inv prob = 10 10 10
tc 9 wred weight = 9 9 9
tc 10 wred min = 48 40 32
tc 10 wred max = 64 64 64
tc 10 wred inv prob = 10 10 10
tc 10 wred weight = 9 9 9
tc 11 wred min = 48 40 32
tc 11 wred max = 64 64 64
tc 11 wred inv prob = 10 10 10
tc 11 wred weight = 9 9 9
tc 12 wred min = 48 40 32
tc 12 wred max = 64 64 64
tc 12 wred inv prob = 10 10 10
tc 12 wred weight = 9 9 9
These are the commands that are currently working under the command line interface:
All of these commands work the same way, averaging the number of packets throughout a specific subset of queues.
Two parameters can be configured for this prior to calling any of these commands:
- qavg n X: n is the number of times that the calculation will take place. Bigger numbers provide higher accuracy. The default value is 10.
- qavg period X: period is the number of microseconds that will be allowed between each calculation. The default value is 100.
The commands that can be used for measuring average queue size are:
The following is an example command with a single packet flow configuration:
./qos_sched -l 1,5,7 -n 4 -- --pfc "3,2,5,7" --cfg ./profile.cfg
This example uses a single packet flow configuration which creates one RX thread on lcore 5 reading from port 3 and a worker thread on lcore 7 writing to port 2.
Another example with 2 packet flow configurations using different ports but sharing the same core for QoS scheduler is given below:
./qos_sched -l 1,2,6,7 -n 4 -- --pfc "3,2,2,6,7" --pfc "1,0,2,6,7" --cfg ./profile.cfg
Note that independent cores for the packet flow configurations for each of the RX, WT and TX thread are also supported, providing flexibility to balance the work.
The EAL coremask/corelist is constrained to contain the default mastercore 1 and the RX, WT and TX cores only.
The Port/Subport/Pipe/Traffic Class/Queue are the hierarchical entities in a typical QoS application:
The traffic flows that need to be configured are application dependent. This application classifies based on the QinQ double VLAN tags and the IP destination address as indicated in the following table.
Level Name | Siblings per Parent | QoS Functional Description | Selected By |
---|---|---|---|
Port | Ethernet port | Physical port | |
Subport | Config (8) | Traffic shaped (token bucket) | Outer VLAN tag |
Pipe | Config (4k) | Traffic shaped (token bucket) | Inner VLAN tag |
Traffic Class | 13 | TCs of the same pipe services in strict priority | Destination IP address (0.0.0.X) |
Queue | High Priority TC: 1, Lowest Priority TC: 4 | Queue of lowest priority traffic class (Best effort) serviced in WRR | Destination IP address (0.0.0.X) |
Please refer to the “QoS Scheduler” chapter in the DPDK Programmer’s Guide for more information about these parameters.