Secure applications with Red Hat OpenShift Kubernetes cluster

In the rapidly evolving landscape of cloud computing, prioritizing the security of your applications is important. This holds especially true in the context of a Red Hat OpenShift Kubernetes Service (ROKS) cluster, where applications might be exposed to the public internet. In this tutorial, explore how to enhance the security of your applications by directing traffic through designated edge nodes within the ROKS cluster. Additionally, learn how to integrate Istio Egress Gateway to effectively manage outbound traffic, offering enhanced control and security measures.

For detailed Istio Egress Gateway configurations and practical use cases, refer to the Git repository.

Addressing the Challenge

By default, applications deployed in a ROKS cluster can be accessible from the public internet. To strengthen security, our objective is to channel all external traffic through edge nodes, thereby bypassing the default worker nodes. This strategy ensures that applications are shielded from direct exposure to potential threats on the public internet.

Integrating Istio Egress gateway

The following figure shows an Openshift cluster, load balancer, and public gateway situated within a virtual private cloud (VPC). Incoming requests initially reach the cluster through the load balancer, then are directed to Ingress gateway pods. These pods then forward the requests to application pods based on Ingress virtual service configurations.

For Egress connections, requests travel from application pods to Egress gateway pods. The Egress pods permit requests based on whitelisted domains using an Egress registry. Following validation, Egress gateway pods authorize requests to external endpoints via the public gateway.

alt

Steps

Step 1. Identifying edge nodes

Log in to your cloud console.
Navigate to the Red Hat OpenShift Kubernetes Service (ROKS) section.
Select your cluster to access the dashboard.
In the ROKS cluster dashboard, navigate to the cluster details section.
Look for a tab or section labeled Nodes or Node Configuration.
Edge nodes are typically labeled or tagged differently from default worker nodes. Identify nodes labeled as Edge nodes or nodes configured to act as the entry point for external traffic.

Step 2. Removing public gateway for default nodes

In the ROKS cluster dashboard, locate the settings or configuration section specifically for the default worker nodes.
This section may be labeled Node Configuration or similar.
Within the node configuration settings, look for options related to network configuration.
Locate the setting for the public gateway associated with default nodes.
Modify the settings to remove or disable the public gateway for default nodes.

Step 3. Configuring routing via edge nodes and Istio Egress gateway

In an OpenShift cluster, Istio Egress gateway plays a crucial role in managing outbound traffic from services within the cluster to external services. It focuses on controlling and routing traffic leaving the cluster. In this example, we are setting Egress gateway to allow outbound connection to ibm.com from our application pods.

If you want to allow a list of different domains for Egress connection, replace ibm.com with the required domain name in the following resources below. Copy and paste the following resources in specific files and apply them using oc apply -f <file-name>.

Gateway: Configured to capture outgoing traffic and direct it to the Egress gateway for processing.

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
 name: istio-egressgateway
spec:
 selector:
istio: egressgateway
 servers:
 - hosts:
- ibm.com
port:
  name: tls
  number: 443
  protocol: TLS
tls:
  mode: PASSTHROUGH

ServiceEntry: Used to define external services that microservices within the cluster need to communicate with.

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
 name: serviceentry
spec:
 hosts:
- ibm.com
 location: MESH_EXTERNAL
 ports:
 - name: tls
number: 443
protocol: TLS
 resolution: DNS

VirtualService: Configured to define routing rules for outgoing traffic, specifying how requests to external services should be processed.

# Below Hostname "istio-egressgateway.istio-system.svc.cluster.local" is given,
# assuming your gateway pods are running in istio-system namespace.
# Else replace "istio-system" based on your setup.
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
 name: ibm-vs
spec:
 exportTo:
 - .
 - istio-system
 gateways:
 - mesh
 - istio-egressgateway
 hosts:
 - ibm.com
 tls:
 - match:
- gateways:
  - mesh
  port: 443
  sniHosts:
  - ibm.com
route:
- destination:
 host: istio-egressgateway.istio-system.svc.cluster.local
 port:
   number: 443
 - match:
- gateways:
  - istio-egressgateway
  port: 443
  sniHosts:
  - ibm.com
route:
- destination:
 host: ibm.com
 port:
   number: 443

DestinationRule: Defines policies for outgoing traffic to external services, influencing how the Egress gateway communicates with those services.

# Below Hostname "istio-egressgateway.istio-system.svc.cluster.local" is given,
# assuming your gateway pods are running in istio-system namespace.
# Else replace "istio-system" based on your setup.
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
 name: dr-egressgateway
spec:
 host: istio-egressgateway.istio-system.svc.cluster.local

Step 4. Implementing network policies

Network policies in Kubernetes define how groups of pods are allowed to communicate with each other and with other network endpoints. To establish network policies for routing traffic through edge nodes, you can create YAML files to specify the desired rules. These rules typically include selectors for identifying pods, Ingress and Egress rules for controlling traffic flow, and podSelector to specify which pods the policy applies to.

Here's an example YAML file illustrating how to define a network policy for routing traffic through edge nodes in a Kubernetes cluster:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: edge-node-policy
spec:
  podSelector:
    matchLabels:
      role: application  # Select pods labeled with "role: application"
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          role: edge-node  # Allow incoming traffic from pods labeled with "role: edge-node"
  egress:
  - to:
    - podSelector:
        matchLabels:
          role: edge-node  # Allow outgoing traffic to pods labeled with "role: edge-node"

In the preceding example YAML file:

apiVersion: Specifies the version of the Kubernetes API to use.
kind: Specifies the type of resource, which in this case is a NetworkPolicy.
metadata: Provides metadata such as the name of the network policy.
spec: Specifies the actual rules for the network policy.
- podSelector: Defines which pods the policy applies to based on their labels.
- policyTypes: Specifies the types of policies allowed, which include Ingress (incoming traffic) and Egress (outgoing traffic).
- ingress: Defines rules for incoming traffic.
  - from: Specifies the source of incoming traffic.
    - podSelector: Selects pods based on their labels.
- egress: Defines rules for outgoing traffic.
  - to: Specifies the destination of outgoing traffic.
    - podSelector: Selects pods based on their labels.

In this example, the network policy allows incoming traffic from pods labeled as role: edge-node and outgoing traffic to pods with the same label. This effectively ensures that traffic is routed through edge nodes labeled as such. Make sure to adjust the labels (role: edge-node, role: application, etc.) according to your specific cluster setup and labeling conventions.

Conclusion

Securing applications in a ROKS cluster from the public internet requires a meticulous approach to routing traffic. By eliminating the public gateway for default nodes, configuring routing through edge nodes, and integrating Istio Egress Gateway, you can significantly enhance the security posture of your applications. Always remember to rigorously test these changes in a controlled environment to guarantee the uninterrupted functionality of your applications while safeguarding them against potential security threats.

Products

Languages

Technologies

All Events

External Resources