Digital Developer Conference: Hybrid Cloud. On Sep 22 & 24, start your journey to OpenShift certification. Free registration

Take a closer look at Istio install artifacts and the external control plane

Istio is an open source service mesh platform that provides a way to control how microservices interact with one another. If you have heard of Istio or have installed Istio, you are probably familiar with the Istio single cluster diagram.

Have you ever wondered what actually gets installed when you install the default profile via the istioctl install command? In this blog, we examine the Istio installation artifacts and discuss how they are used with the default Istio installation profile and with the new external control plane model that was added to Istio 1.7.

What’s in the default Istio install?

The following diagram shows how resources are deployed in the cluster after you complete the default installation of Istio:

Image showing the 3 layers of Istio installation

Let’s take a closer look at exactly what is in those layers:

Deployment and services artifacts

  • Istiod: Istiod is the kernel for the Istio control plane which provides a Certificate Authority (CA) server, an Envoy xDS server and webhook servers. There are also multiple Istio configs like the ones listed below that ensure Istiod is bootstrapped properly and able to securely communicate to the sidecar proxies in the mesh.
  • Gateway: Istio ingress gateways control the inbound traffic to your mesh.

Configs

  • Custom Resource Definitions (CRDs): Various Istio CRDs such as Gateway, Virtual Services, Destination Rules etc.
  • ConfigMaps: There are two main ConfigMaps which are used to store info about mesh and injection template configurations separately. You can edit these two ConfigMaps if you need to change the default mesh runtime configuration or you need to use your own injection template.
  • Admission webhook configurations: Configurations for two Istio webhooks, the validation webhook and sidecar injector webhook. They include trigger rules, failurePolicies, webhook server addresses as well CA bundles which the webhook clients need when connecting to Istiod running as webhook servers.
  • Service account, service role binding, etc: This makes sure Istiod has the right access authorities for different kinds of resources, such as namespaces, services, custom resources, configmaps etc.
  • Secrets: store key and certs and credentials related to service account or Istiod. Istiod saves its root cert into Istio-ca-secret if no external cert is plugged in, or a user can plug in his own root cert under cacerts secret. The root cert is used internally by Istiod as well; for example it is used to patch CA bundle for webhook configurations.

Istiod reads information from these Istio configs to finish its initiation and then start the CA, xDS and webhook servers. Once the sidecar proxies in the mesh are connected to Istiod, Istiod starts to push configurations to the sidecar proxies.

A mesh admin interacts with Istio via ConfigMaps or via Istio mesh-wide resources such as authentication policy or edge configurations such as gateway resources. Service owners interact with Istio via namespace or service-level configurations such as virtual services or destination rules or namespace/service level authentication policy.

External control plane deployment model

Istio 1.7 introduced the external control plane deployment model for mesh operators and mesh admins. In this model, Istiod runs on an external cluster, as the diagram below shows. To learn more about why we introduced this model and how it can provide a much simplified experience for mesh admins, read our Deploying Istio Control Planes Outside of the Mesh blog.

external control plane diagram

Let’s look closer at this deployment model. In this setup, mesh operators should set up the Config/Remote Cluster first with the istiodRemote and base components. Then, mesh operators should install Istiod on the External Control Plane Cluster, with its KUBECONFIG configured to the Config/Remote Cluster via the istio-kubeconfig secret. The Config/Remote Cluster serves as not only the config cluster because istiod reads the configuration from the cluster, but also the remote cluster because it also runs users’ services.

Here is a sample snippet mesh operators can use to install lightweight Istio on the Config/Remote cluster:

apiVersion: install.istio.io/v1alpha1 
kind: IstioOperator 
spec: 
  addonComponents: 
    prometheus: 
      enabled: false 
  components: 
    base: 
      enabled: true 
    ingressGateways: 
    - enabled: true 
    istiodRemote: 
      enabled: true 
    pilot: 
      enabled: false

When mesh operators deploy the above sample snippet using istioctl, the command essentially instructs the istioctl to install the base and istiodRemote component on the cluster. Note that pilot is disabled, which basically configures the Istio installer to not install Istiod on the cluster.

Below is a sample snippet mesh operators can reference to install Istiod on the external control plane cluster. No CRD and webhook configurations are installed on the external control plane cluster, so there is no need to install the base component there.

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    base:
      enabled: false
    ingressGateways:
    - enabled: false
    pilot:
      enabled: true
values:
  global:
    operatorManageWebhooks: true

When a mesh operator deploys the above sample snippet using istioctl onto the external control plane cluster, it instructs the istioctl to install the Istiod control plane (pilot component) in the cluster and not install the base or any of the webhook configurations. Mesh operators who are interested in exploring this further, can follow the external istiod single cluster step by step guide to try this out.

The diagram above shows a single cluster as the data plane for an Istio mesh. A mesh admin can expand the remote cluster to multiple clusters, all managed by the same Istiod running in the external control plane cluster.

Note, one of the remote clusters also acts as a config cluster and a mesh admin only needs to make mesh-wide configurations to the config cluster. Similarly, a mesh admin and user only needs to deploy Istio resources to the config cluster. External istiod will read the configuration from the config cluster and push them to all sidecar proxies in the config cluster and remote cluster(s). Remote clusters don’t have CRDs installed, unlike the config cluster. The following image shows this configuration:

External istiod reads configuration from the config cluster and pushes them to sidecar proxes

Mesh operators can further expand this deployment model to manage multiple Istio mesh from an external control plane cluster that runs multiple Istiods, as the diagram below shows

Managing multiple Istio mesh from an external control plane

Mesh operators can host multiple Istiods on the external control plane cluster and each Istiod manages its own config/remote cluster. Mesh operators can install their own Istio mesh (for example, the minimal profile or default profile) in the external control plane cluster and configure its Istio ingress gateway to route traffic between config/remote clusters to their matching istiod deployments.

The ingress gateway can perform TLS passthrough on the ports needed by Istiod (currently 15012 for xDS server and 15017 for webhook servers) so the communication from sidecar proxies in the config/remote cluster to their istiod can be continuously secured via mutual TLS. Mesh operators can refer to these steps to learn more about how to configure their Istio ingress gateway.

Conclusion

We are excited to introduce this external control plane model in Istio 1.7. This allows vendors like IBM to provide Istio-as-a-Service to our Istio mesh admins without the mesh admins having to worry about running and managing the Istio control plane. This further simplifies the operation for the mesh admin, thus freeing the mesh admin to focus on working with the service owners to onboard services to the mesh and leveraging Istio to serve their business needs.