Now available! Red Hat OpenShift Container Platform for Linux on IBM Z and LinuxONE Learn more

IBM Power Systems SR-IOV ACL on Mellanox device drivers

Introduction

Access control list (ACL) rules in single root I/O virtualization (SR-IOV) allow traffic classifications and filtering. It can enforce the security policy by restricting the allowed virtual machine (VM) traffic and filtering it which prevents most L2 attacks. In this article, we explain how to create and validate the virtual functions (VFs) with the ACL rules on the Mellanox device driver.

What are access control lists?

An ACL is a network filter used by adapters to permit and restrict data flows into and out of network interfaces. When an ACL is configured on an interface, the network adapter analyses the data passing through the interface, compares it to the criteria described in the ACL, and either permits the data to flow or prohibits it.

Why do we use ACLs?

There are a variety of reasons to make use of ACLs. The primary reason is to provide a basic level of security for the network.

Why do you need to use SR-IOV ACLs?

A single root I/O virtualization (SR-IOV) adapter provides I/O virtualization capabilities for a managed system. The APIs provide the configuration of the SR-IOV adapters.

A Peripheral Component Interconnect Express (PCIe) VF is a lightweight PCIe function on a network adapter that supports SR-IOV. The VF is associated with the PCIe physical function (PF) on the network adapter and represents a virtualized instance of the network adapter. Each VF has its own PCI configuration space.

ACL rules allow traffic classifications and filtering. SR-IOV with a combination of ACL can provide security by restricting the allowed traffic and filtering it.

SR-IOV is supported in the following Mellanox adapters:

  • Haleakala 2-Port 100Gb Gen4 PCIe (FC: EC66)
  • Everglades 2-port 10Gb (FC: EC2S)
  • Everglades 2-port 25Gb (FC: EC2U)
  • Glacier Park 2-port 100Gb (FC: EC3M)

ACLs in network device drivers

In network device drivers, ACLs are implemented using the Media Access Control (MAC) address, and virtual local area network (VLAN) restrictions at the hypervisor and OS level.

Figure 1 describes how the physical adapter is virtualized using the power hypervisor and how the virtual functions are assigned to the IBM® AIX® partitions.

Figure 1. SR-IOV deployment on Power Systems

fig1

ACL commands on AIX

Attributes used to set the MAC and VLAN restrictions:

chhwres -r sriov -m <Managed_System> --rsubtype logport -p <Lpar_Name> -o a -a "adapter_id=<Adapter_ID>,phys_port_id=<Physical_Port_ID>,logical_port_type=eth, allowed_os_mac_addrs=<MAC_Options>, allowed_vlan_ids=<VLAN_Options>"

For MAC restrictions, we use allowed_mac_os_addrs with the following options:

  • none
  • all
  • Allowed list in format 023344556677 (it should start with 02)

For VLAN restrictions, we can use allowed_vlan_ids with options as [2,3,4,5,…4095].

ACL command usage

This section provides examples to create MAC and VLAN ACLs.

Command to create a VF with allowed MAC and allowed VLAN:

chhwres -r sriov -m p9zzh-fsp --rsubtype logport -p p9zzh-cx5a -o a -a "adapter_id=2,phys_port_id=0,logical_port_type=eth, promisc_mode=0, allowed_os_mac_addrs=024444555544, allowed_vlan_ids=4, port_vlan_id=0"

Command to create a VF with only allowed MAC:

chhwres -r sriov -m p9zzh-fsp --rsubtype logport -p p9zzh-cx5a -o a -a "adapter_id=2,phys_port_id=0,logical_port_type=eth, promisc_mode=0, allowed_os_mac_addrs=024444555544, port_vlan_id=0"

Command to create a VF with only allowed VLAN:

chhwres -r sriov -m p9zzh-fsp --rsubtype logport -p p9zzh-cx5a -o a -a "adapter_id=2,phys_port_id=0,logical_port_type=eth, promisc_mode=0, allowed_vlan_ids=4, port_vlan_id=0"

Checking ACLs on IBM AIX

We check ACL values on a VF on AIX using the entstat utility available for the adapter.

The entstat -d entX command gives the following output specific to the allowed MAC & VLAN list:

Port VLAN: Disabled
MAC Mode: Restricted To List
Maximum Active MAC Addresses: 1
Allowed MAC Addresses:
    F2:5F:A4:F5:CA:00 -> Default MAC Address
    02:44:44:55:55:44 -> MAC address from the list of allowed macs
VLAN Mode: Restricted To List
Maximum Active VLAN IDs: 20
Allowed VLAN IDs:
         0004       -> VLAN from the list of allowed vlans
Enabled VLAN IDs: None

Restrictions

The following restrictions are applicable for MAC and VLAN ACL options while creating a VF:

  • Up to four MAC addresses are allowed in the MAC list.
  • Up to 20 VLAN IDs are allowed in the VLAN list.
  • If the VLAN ID Restriction option is set to Allow List, then for each VLAN ID in the list the operating system intends to have active on the interface at the same time there must be an adapter VLAN filter available.
  • When Port VLAN ID on a VF is greater than 1, the allowed_vlan_ids attribute is set to None.

Design of ACL for Mellanox

For a Mellanox adapter: allowed_mac_addrs is independent of allowed_vlan_ids while creating a VF. Refer to the following table for the combination of the ACLs on OS and their expected results.

ACL rules in Mellanox device drivers

Table 1 shows the expected results when the VF is created with ACL.

Table 1. ACL rules in Mellanox device drivers

S.No HMC command options OS Expected results
MAC VLAN MAC VLAN Ping on base adapter Ping on VLAN pseudo interface
1 ALL (Default) ALL (Default) Default NO VLAN Sent NA
2 Default VLAN-X Sent Sent
3 MAC-X NO VLAN Sent NA
4 MAC-X VLAN-X Sent Sent
5 ALL NONE Default NO VLAN Sent NA
6 Default VLAN-X Sent VLAN config fails
7 MAC-X NO VLAN Sent NA
8 MAC-X VLAN-X Sent VLAN config fails
9 ALL Allow Default NO VLAN Dropped NA
10 Default VLAN-X Dropped Sent
11 MAC-X NO VLAN Dropped NA
12 MAC-X VLAN-X Dropped Sent
13 MAC-X VLAN-Y Dropped VLAN config fails
14 NONE ALL Default NO VLAN Sent NA
15 Default VLAN-X Sent Sent
16 Default VLAN-Y Sent Sent
17 MAC-X NO VLAN Change MAC fails NA
18 MAC-X VLAN-X Change MAC fails NA
19 NONE NONE Default NO VLAN Sent NA
20 Default VLAN-X Sent VLAN config fails
21 MAC-X NO VLAN Change MAC fails NA
22 MAC-X VLAN-X Change MAC fails NA
23 NONE Allow Default NO VLAN Dropped NA
24 Default VLAN-X Dropped Sent
25 MAC-X NO VLAN Change MAC fails NA
26 MAC-X VLAN-X Change MAC fails NA
27 Default VLAN-Y Dropped VLAN config fails
28 Allow ALL Default NO VLAN Sent NA
29 Default VLAN-X Sent Sent
30 MAC-X NO VLAN Sent NA
31 MAC-X VLAN-X Sent Sent
32 MAC-Y NO VLAN Change MAC fails NA
33 Allow NONE Default NO VLAN Sent NA
34 Default VLAN-X Sent VLAN config fails
35 MAC-X NO VLAN Sent NA
36 MAC-X VLAN-X Sent VLAN config fails
37 MAC-Y NO VLAN Change MAC fails NA
38 Allow Allow Default NO VLAN Dropped NA
39 Default VLAN-X Dropped Sent
40 MAC-X NO VLAN Dropped NA
41 MAC-X VLAN-X Dropped Sent
42 MAC-Y NO VLAN Change MAC fails NA
43 MAC-X VLAN-Y Dropped VLAN config fails

Table 2 explains the terms used in Table 1.

Table 2. Description of ACL terms
Term Description
MAC-X Allowed MAC
MAC-Y Not allowed MAC
VLAN-X Allowed VLAN
VLAN-Y Not allowed VLAN
Sent Ping success
Dropped Ping failed
NA Not Applicable

Summary

ACL is a principle element in securing your networks, and therefore, understanding their function and proper placement is essential to achieve best results.

Swetha Venkannagari
Neelima Nandula