Enhanced Transmission Selection (ETS) is one of the Data Center Bridging Exchange (DCBX) standards, a quality of service (QoS) feature through which each traffic class, such as Ethernet, Fibre Channel (FC), and so on, has a minimum allocated bandwidth. ETS is an algorithm that works based on strict priority queues and bandwidth-assigned queues defined by adapter. Each traffic class has a bandwidth defined by the user.

Converged Ethernet adapters are capable of sending multiple types of traffic.

Each traffic type requires a set of priorities, which are in line with the traffic type characteristics. Using the Institute of Electrical and Electronics Engineers (IEEE) protocol, VLAN priority code point (PCP) can be used to define the priority of a traffic type. VLAN PCP can provide a priority across multiple traffic types but cannot avoid the interference of one traffic over the other traffic. Using ETS, it is possible to isolate traffic types.

This tutorial explains how to configure the ETS feature on network adapters in IBM AIX.

Why ETS provides better quality of service (QoS)?

The ETS algorithm provides a way to:

Allocate a guaranteed share of bandwidth to each traffic class.
Allow each traffic class to exceed this minimum guaranteed bandwidth if other traffic classes are not utilizing their share.

ETS achieves this by leveraging VLAN priority code point (PCP), defined under the IEEE 802.1p standard, to determine type of traffic.

ETS terms

The following list describes some of the ETS terms used in this tutorial:

Priority: There are eight traffic priorities available, and no two traffic classes can share the same priority
Traffic class bandwidth percentage (TC%): Percentage of available link bandwidth allocated to a particular traffic class.

Prerequisites for using ETS with AIX

Hardware-specific prerequisites:

In IBM AIX, ETS configuration is inherited from the network switch. And therefore, users must configure ETS on the switch. Switch advertises this ETS configuration to its peers (the adapters in the AIX logical partition), that are connected to the switch. In this way, the adapter inherits the ETS configuration that is performed on the switch.
Adapters should be in the dedicated mode and no Single Root I/O Virtualization (SR-IOV) virtual functions (VFs) should be assigned to the LPAR as it is not supported.

Make sure to use any of the following supported adapters only.

Feature codes	Description
EC3L and EC3M	PCIe3 2-port 100 GbE RoCE QSFP28 PCIe 3.0 x16 adapter
EC2R and EC2S	PCIe3 2-pORT 10 Gb NIC and ROCE SR/Cu adapter
EC2T and EC2U	PCIe3 2-pORT 25/10/1 Gb NIC and ROCE SFP28 adapter
EC66 and EC67	PCIe4 2-port 100 GbE RoCE x16 adapter
EC75 and EC76	PCIe4 2-port 100 GbE RoCE x16 adapter
EC71 and EC72	PCIe4 2-port 25/10/1 Gb RoCE SFP28 adapter
EN24 and EN26	PCIe4 x16 4-port 25/10/1 GbE RoCE SFP28 adapter

Software-specific prerequisites:

Make sure that the following software prerequisites are met to use ETS with AIX:

Adapter should not be part of any EtherChannel Link Aggregation Control Protocol (LACP) load balancing or teaming mode.
VLAN adapters must be created on top of the physical adapter. (More details follow in next section). Regarding the adapter tunable changes for configuring ETS, make sure to set ets_mode to willing.

Sample demonstration of using ETS in AIX

Perform the following steps to enable ETS in AIX:

Perform switch-side configuration. In this example, NVIDIA MSN3420-CB2R with Onyx 3.9.3220 network switch has been used.
1. Configure the traffic class along with switch priority. Multiple traffic classes can be configured with different priorities (ranging from 0 to 7).
```
interface ethernet <port> traffic-class <0-7> bind switch-priority <0-7>
```
2. Assign the required bandwidth to the configured traffic class.
```
interface ethernet <port> traffic-class 0 dcb ets wrr <0-100>
```
3. Show switch priorities assigned to different traffic classes.
```
show qos interface ethernet <port> tc-mapping
```
Perform AIX-side configuration. In this example entX is physical adapter from the above ETS supported list of adapters.
1. Enable ETS on adapter by changing the tunable attribute, ets_mode to willing. In the following command, entX is the adapter or the device and chdev sets the device attribute ets_mode to willing.
```
chdev –l <entX> -a ets_mode=willing
```
2. Configure the VLAN along with the priorities. It is possible to configure up to a maximum of eight VLANs with different priorities. In the following command, mkdev creates a VLAN and entX represents the adapter.
```
mkdev –c adapter –s vlan –t eth –a base_adapter=’<entX>’ -a vlan_tag_id=’<vlanid>’ -a vlan_priority=’<0-7>’
```
Perform validation of ETS on AIX. To check if AIX has properly inherited the ETS settings on the switch, run the entstat command on the network adapter. It displays the inherited ETS configuration from the switch.
```
entstat –d <entX>
```
In the following example, different traffic classes are set to different bandwidths on the switch:
- TC0 having priorities 0, 1, and 2 is set to 60% bandwidth
- TC1 having priorities 3, 4, and 5 is set to 25% bandwidth
- TC3 having priorities 6 and 7 is set to 15% bandwidth
Operational configuration refers to both switch and adapter agreed configuration.

Remote side configuration represents switch-side ETS configuration.
Optionally, if required, disable ETS on the adapter.
```
chdev –l <entX> -a ets_mode=disabled
```

Detailed explanation of ETS configuration

As shown in the following figure, there are three traffic classes with different bandwidth allocated that are guaranteed. Sum of all bandwidths of traffic classes should be 100%.

The following section provides an explanation of this with examples.

Examples for bandwidth scaling of traffic class

Scenario 1

In this scenario, three traffic classes are configured with each traffic class assigned bandwidths as 60% (TC0), 25% (TC1), and 15% (TC2). This is then bombarded with some heavy traffic over TC0 and TC2. As TC1 is inactive, you can notice that TC0 and TC2 are scaling beyond the assigned bandwidth by utilizing the unused bandwidth share of TC1.

The following configuration indicates that:

TC0 is set to 60% bandwidth with priorities 0,1, and 2.
TC1 is set to 25% bandwidth with priorities 3, 4, and 5.
TC2 is set to 15% bandwidth with priorities 6 and 7.

The following screenshot shows the statistics of bandwidth scaling when TC1 was inactive (that is, no traffic is bombarded on TC1). TC1 bandwidth is shared equally between TC0 and TC2.

Scenario 2

The configuration set in scenario 1 is used in this scenario as well. Here, TC1 is activated (that is, traffic is bombarded on TC1). As a result, it is observed that TC1 is able to transmit 25% of its assigned bandwidth immediately after becoming active.

The following screenshot indicates that:

TC0 is able to transmit 60% bandwidth.
TC1 and TC2 are able to transmit 25% and 15% bandwidth respectively.

Scenario 3

The configuration set in scenario 1 is used in this scenario as well. Here, both TC0 and TC1 are made inactive (that is, no traffic is bombarded on TC0 and TC1). As a result, it is observed that, the remaining TC2 traffic class is able to scale up to 100% of its bandwidth by utilizing the unused bandwidth share of the other two traffic classes.

The following screenshot shows this statistics.

Summary

This tutorial explained the basic concepts of the ETS feature and its support in AIX. It also provided the steps to configure and validate ETS easily in AIX.

Configure the Enhanced Transmission Selection (ETS) feature in IBM AIX

Configure the Enhanced Transmission Selection (ETS) feature in IBM AIX