Constructing and enhancing IBM's software-defined networking (SDN) solution for the public cloud has been an intriguing journey. This ongoing effort has tackled various technical challenges from diverse perspectives. In this article, we reflect on our experiences and offer insights into the path we have traversed.

Requirements and architecture paradigms

We begin by providing a summary of the requirements and the main architectural paradigms. Our focus lies on the SDN stack of endpoints for hypervisors and virtual bare metal, integral components of the network virtualization solution.

Figure: A hypervisor or bare-metal machine connected to the DC network via a NIC

Specifically, we explore the packet processing aspect of network virtualization implementation, emphasizing the intricate interplay between hardware offloading and software processing across different traffic flows and various vendors of network interface cards (NICs).

Distributed implementation

From a high-level perspective, our SDN solution needed to meet several key requirements: supporting a cloud-scale SDN with unpredictable customer traffic patterns/use cases, ensuring full isolation between tenants, and maximizing available bandwidth and performance for customer traffic. To address these requirements, we established the following architectural principles:

Endpoint-centric processing: We adopted a distributed paradigm, eliminating middle boxes and pushing virtual network processing to endpoints.
Hardware offloading: Our aim was to offload as much packet processing as possible to NIC hardware.
Interface flexibility: Guests were given the freedom to choose any virtual interface type, whether Virtio, SR-IOV, accelerated vNICs (vDPA), or a VLAN device.

Endpoint-centric processing

To realize endpoint-centric processing, we employed a "bump in the wire" hardware approach. We leveraged software pipeline processing as the slow path, while directing most traffic to the smart NIC hardware pipeline as the fast path.

An example of this approach is implementing virtual router functionality in a distributed manner (known as distributed virtual routing (DVR)) at the endpoint, eliminating the need for a single component in the system to function as the virtual router. This distributed pattern of network virtualization, extending to functionalities such as virtual routers, network address translation (NAT), and load balancing (LB), is a widely adopted industry practice. It facilitates scalability and high performance without a single point of failure.

Hardware offloading

Our design principle of hardware offloading necessitated additional processing power at the endpoints to execute network processing on each packet. To minimize computation cycles on the hosts, we utilized programmable NICs to offload network virtualization processing to the NIC hardware. This approach reduces host CPU consumption and provides benefits such as lower latency, low jitter, and high throughput. We integrated multiple vendor and in-house technologies into our network gear, with the goal of basing our SDN solution on any third-party smart NIC vendor or in-house hardware solution.

Adaptation to NIC capabilities

Throughout our ongoing efforts, we encountered changes in NIC hardware capabilities and emerging technologies, such as P4-based NIC architecture. While adhering to our principle of offloading computation to the NIC, we adapted to the varying capabilities of each NIC vendor used in our SDN solution.

An essential capability we needed to adapt to was the supported NIC hardware rule scale, which refers to the maximum number of hardware rules and the hardware update rate supported by the NIC. Initially, we utilized wildcard hardware-rules matching to reduce the total number of rules, avoiding a one-to-one relationship for session-to-hardware rule pairs. However, we discovered that wildcard-based hardware pipeline matching led to performance degradation compared to exact-match hardware rules for most vendors.

Over time, NIC vendors made incremental improvements to hardware rule scaling and introduced additional capabilities. We also observed a shift towards more agile technologies to support new hardware offloading capabilities in smart NICs, such as P4 hardware pipelines. Smart NIC hardware vendors successfully increased both the total number of supported hardware rules and the hardware update rate.

Hardware pipeline abstraction

In our architectural pipeline design, vendor agnosticism stands out as a primary requirement. We aimed to maintain a unified SDN solution independent of the specific NIC hardware deployed on each platform. To achieve this, we needed a generic abstraction enabling us to maintain our SDN stack solution while maximizing the utilization of each NIC's capabilities. Essentially, regardless of the NIC hardware deployed, our solution aimed to optimize hardware offloading resources while maintaining a unified SDN stack solution agnostic to the NIC hardware.

To accomplish this goal, we developed a generic flow-based abstraction. This abstraction provides a unified interface to the SDN stack, maximizing hardware offloading capabilities regardless of the specific NIC vendor, technology, or supported API. Supported APIs could include TC Flower, RTE Flow, P4 Portable NIC Architecture (PNA), or any other vendor-specific API, such as emerging P4-based smart NICs pipelines/SDKs.

Our exploration revealed that flow-based exact match can accommodate all possible use cases in the cloud while delivering advanced hardware offloading performance. It's adaptable to all hardware vendors. Initially, we utilized wildcards in the hardware pipeline design to reduce the total number of required hardware rules. Transitioning to a flow-based approach, we developed a mechanism enabling flexible decisions during hardware offloading based on smart NIC capabilities, flow use cases (e.g., east-west vs. north-south NAT), and flow characteristics (e.g., mice vs. elephants).

Offloading modes: Inline vs. offline

In consideration of actual offloading, we evaluated two primary approaches: inline, processing packets in software, or offline, using a dedicated worker/CPU to manage hardware offloading. Given the requirement to maintain a multi-vendor solution and intelligent offloading decision-making, we opted for the offline approach, managing hardware offloading with a dedicated worker.

Figure: Inline vs. offline offloading modes

Offloading criteria configuration

We devised a configuration for offloading criteria analyzed in runtime while processing packets using an offload predictor module. This module enables us to define during production which flows should be offloaded and when. Initially, we implemented a simple criteria based on the session lifetime or the packets count within a time threshold (PPS). For instance, we optimized the system to offload all RDMA AI training workloads (e.g., RoCE packets) without delay, configuring high QoS class types to achieve ultra-low latency. In contrast, for short lived mice flows, we avoid offloading to hardware and instead continue processing in software. Subsequently, we added more advanced criteria based on learned customer traffic patterns for each traffic use case.

Summary

This article provided an overview of the software-defined network (SDN) solution for the IBM Cloud, with a specific focus on the interface between the software stack and the hardware NIC. This interface plays a crucial role in the successful implementation of the SDN solution, enabling it to achieve top performance and flexibility with the hardware it interacts with.

The interface between the software stack and the hardware NIC is important for the SDN solution's success. Leveraging vendor-agnostic support and granular offload configuration ensures compatibility across different hardware setups. The IBM Cloud SDN solution caters to a wide range of customer use cases, spanning from AI training workloads over virtualized RoCE networks, HPC workloads, and cloud applications, to virtualized BM and hybrid cloud use cases. Leveraging the vendor-agnostic flow-based abstraction, along with finely tuned offloading criteria, enabled us to implement a single SDN solution across all these use cases.

Call to action

Based on our experience, we advocate for the establishment of a standardized, flow-based API in the industry that all hardware (HW) vendors can support. Our experience underscores that such an API has the capability to accommodate various traffic types that are used in the cloud, all while delivering high performance.

Building a vendor-agnostic flow-based SDN for IBM Cloud

Building a vendor-agnostic flow-based SDN for IBM Cloud