Kubernetes with OpenShift World Tour: Get hands-on experience and build applications fast! Find a workshop!

Secure communication between services in Istio with mutual TLS

As organizations continue their digital transformation journey, one of the main factors that helps them is the ability to build and deploy solutions quickly.

Solutions built on a microservices-based architecture help solve the problem of breaking down complex tasks into manageable portions that can be built and deployed independently. However, managing a growing set of microservices across different business units within organizations can quickly become a huge headache, if not done well. A service mesh framework like Istio provides the following advanced features to help improve availability and resiliency:

  • observability through rich tracing, monitoring, and logging of services
  • traffic management through timeouts, retries
  • paths for safe upgrades through canary rollouts and A/B testing
  • automatically secured services by enabling end user and transport authentication, authorization, and audit capabilities

One of the main challenges of managing microservices-based solutions is how to properly secure not just the microservices themselves but also the communication between them and the access to these services by different kinds of users and external services. In a microservices-based architecture, you can implement security at the application layer, which includes architecting identity management and various authentication mechanisms to verify the identity of users and services into the application code itself. This approach can quickly become painful to manage if you take into account scaling out these services. Here is where a service mesh technology like Istio can help. It uses Envoy’s sidecar proxies to intercept network traffic flowing to and from services and securing communication.

This tutorial focuses on how Istio manages security within a service mesh, specifically on how to use mutual transport layer security (TLS) to secure communication between services. Istio supports two types of authentication:

  • Transport authentication, which provides service-to-service authentication. (Istio supports only mutual TLS for transport authentication at this time.)
  • Origin authentication (also known as end-user authentication), which provides client-to service authentication. (Istio supports only JSON Web Tokens for origin authentication at this time.)

Prerequisites

To complete this tutorial you need to set up the following environment:

Estimated time

Completing this tutorial takes about 20 minutes, after the prerequisites are set up.

Understanding how mutual TLS works with Istio

TLS, a protocol designed to provide secure communication between apps, supports many algorithms to exchange keys and verify message integrity, and various ciphers to encrypt messages. As the number of services scales across multiple deployments, securing them properly can be a daunting task.

Istio completely shifts the burden of configuring security for each individual service away from developers. Istio’s Citadel component (and other components like Envoy sidecar proxies, Pilot and Mixer) manages all the parts and pieces of securing the services in a service mesh. Istio supports mutual TLS, which validates the identify of both the client and the server services.

The Citadel component in Istio manages the lifecycle of keys and certificates issued for services. When Istio establishes mutual TLS authentication, it uses these keys and certificates to exchange the identities of services. To establish a mutual TLS connection between two services, the envoy proxy on the client side establishes a mutual TLS handshake with the envoy proxy on the server side during which the client side envoy proxy verifies the identity of the server side and whether it is authorized to run the target service. When the identities of the services are verified, the mutual TLS connection is established and the client service sends communication through the client side proxy to the server side proxy and finally to the target service.

Services use authentication policies to define the kind of requests that a service receives, whether it is encrypted using mutual TLS or plain text. Istio uses these authentication policies, along with service identities and service name checks, to establish mutual TLS connection between services. The authentication policies and secure naming information is distributed to the Envoy proxies by the Pilot component. The Mixer component handles the authorization and auditing part of Istio security.

The following sections walk through the process of enabling mutual TLS connections between services in Istio. You need to define a Policy object and DestinationRule object. You use a Policy object (also called an authentication policy) to define what kind of requests a service receives. A DestinationRule object applies to the traffic that is destined for a target service. It tells the client services whether to send encrypted traffic to the target service or to send plain-text requests.

To enable a mutual TLS connection between services, you need to define a Policy object and a DestinationRule object. However in the Istio 1.4, a new automatic mutual TLS feature was added. If you turn on this setting, services are automatically enabled with mutual TLS, and you only need to specify a Policy object (a DestinationRule object is not needed). However, this tutorial details how to define the Policy object and a DestinationRule object to enable mutual TLS between services.

Authentication policy

You can apply a Policy object to specific services as defined in the targets section. Or, you can apply it to a wider scope, for example, to all services in a namespace or a mesh wide scope.

Service-specific policy

Review the following example of a service-specific policy:

apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: “bar-policy”
spec:
   targets:
   - name: bar
   peers:
   - mtls: {}

This example shows the following information:

  • The kind key defines the configuration object you are creating (in this case, an authentication policy).
  • The targets key defines the services that this policy applies to.
  • The peers key defines the authentication mechanism to use and any additional parameters needed. Istio currently supports only mTLS as the transport authentication mechanism.

Mesh-scoped policy

Review the following example of a mesh-scoped policy:

apiVersion: "authentication.istio.io/v1alpha1"
kind: "MeshPolicy"
metadata:
  name: default
spec:
   peers:
   - mtls: {}

This example shows a MeshPolicy definition that applies to all services in the service mesh scope. There can be only one MeshPolicy defined for a service mesh, and its name should be default, with no targets key specified.

Namespace-scoped policy

Review the following example of a namespace-scoped policy:

apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: “default”
  namespace: “ns1”
spec:
   peers:
   - mtls: {}

This example shows a namespace-scoped policy definition that applies to all services in the namespace, as specified in the namespace key. There can be only one namespace-scoped definition for a namespace, with no targets key specified. Because no targets key is specified, the name must be default.

PERMISSIVE and STRICT modes

Review the following example of a PERMISSIVE/STRICT mode:

apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
  name: “bar-policy”
spec:
   targets:
   - name: bar
   peers:
   - mtls: {}
     mode: PERMISSIVE

This example shows a policy definition for a specific service, as specified in the targets key. According to the policy definition for the bar-policy service (peers.mode: PERMISSIVE), it allows both plain text and encrypted traffic from clients. You can change it to allow only encrypted traffic by changing peers.mode from PERMISSIVE to STRICT.

For services with transport authentication enabled by an authentication policy, the peers section has an additional key (mode), which you can use to define what kind of traffic a service can accept from its peers. This mode key can take two values: PERMISSIVE or STRICT. If PERMISSIVE is set, the service can accept both plain-text traffic as well as encrypted. If STRICT is set (the default value for the mode key is STRICT), the service only accepts mutual TLS traffic.

In Istio, you need to install sidecar proxies for each service if you want to establish mutual TLS communication. There might be cases during an onboarding process when the operator cannot install sidecar proxies for all client services at the same time. In these situations, you still want to enable communication between non-Istio client services and Istio target services. By enabling the permissive mode in the authentication policy for a target service, non-Istio client services can continue to send plain-text traffic to the target service until the onboarding process is complete. Then you can switch to the strict mode.

DestinationRule object

Mutual TLS is about securing traffic in both directions from a source service to a target service and back. Use a Policy object to instruct a target service to receive mutual TLS traffic. Use a DestinatonRule object to instruct client services to make a mutual TLS connection with a target service using the required certificates. DestinationRule objects are an important part of Istio’s traffic management policy, which configures what happens to the traffic meant for a given destination or target service. There are several TLS settings that you can configure in a DestinatonRule to enable mutual TLS communication with a destination service.

You can use the spec.host key to specify the destination service for which the TLS setting needs to be configured. Then the client services know to send the appropriate certificates required by the destination service.

You can set the tls: mode key of the trafficPolicy section in a DestinationRule to enable or disable the TLS connection to a destination service using one of the following settings: SIMPLE, MUTUAL, ISTIO_MUTUAL, or DISABLE. The TLSSetting.TLSMode of the DestinationRule object provides a clear definition of all the different settings possible.

Steps for deploying a sample application

This section walks through deploying the sample Bookinfo app on an Istio installion using istio-demo.yaml (without mutual TLS enabled). You enable mutual TLS for the services in the Bookinfo app by creating a Policy object and a DestinationRule object. This example deploys Istio on a Kubernetes cluster running on IBM Cloud .

  1. Install Istio without mutual TLS enabled. (Use istio-demo.yaml or set the global.mtls.enabled installation option to false.)
  2. Deploy the Bookinfo application in the default namespace:

    kubectl label namespace default istio-injection=enabled
    
    kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
    
  3. Check the status of the services and pods for the Bookinfo app:

    kubectl -n default get pod,svc

  4. Check the mesh policy for the default namespace before making any changes:

    kubectl get MeshPolicy default -o yaml

    The output should be something similar to the following example. (Make sure mTLS mode is PERMISSIVE to enable the services to receive plain-text and/or encrypted traffic.)

     apiVersion: authentication.istio.io/v1alpha1
     kind: MeshPolicy
     metadata:
       annotations:
       ….
     spec:
       peers:
       - mtls:
          mode: PERMISSIVE
    
  5. Verify that you can access the product page:

     kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o “<title>.*</title>"
    
     <title>Simple Bookstore App</title>
    
  6. To see reviews run the following command:

     kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | sed -n '/<blockquote>/,/<\/blockquote>/{ /blockquote>/d; p }’
    

    The output should look like the following example:

     <p>An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!</p>
    
     <small>Reviewer1</small>
    
     <p>Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.</p>
    
     <small>Reviewer2</small>
    
  7. Check if there are any DestinationRule objects applied to this namespace:

    kubectl -n istio-system get DestinationRule default -o yaml
    

    The result should look similar to the following example:

    Error from server (NotFound): destinationrules.networking.istio.io "default" not found

  8. Enable mTLS by defining an authentication policy (meshwide or namespace or specific service) for bookinfo services. The following example defines a Policy object for the reviews service:

     cat <<EOF | kubectl apply -n default -f -
     apiVersion: "authentication.istio.io/v1alpha1"
     kind: "Policy"
     metadata:
       name: "reviews"
     spec:
       targets:
       - name: reviews
       peers:
       - mtls: {}
     EOF
    
  9. Verify that the Policy object was created:

     kubectl get Policy -n default
    
  10. So far you only enabled the authentication policy for the reviews app. This means that the reviews service only receives encrypted traffic, but the clients sending requests to this service do not know and continue to send plain-text traffic to the reviews service. So if you try to get the reviews of the books from the Bookinfo app, you won’t see any results returned:

     kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl http://productpage:9080/productpage | sed -n '/<blockquote>/,/<\/blockquote>/{ /blockquote>/d; p }'
    

    Because the details app does not have mTLS enabled, you should still be able to get the title of the book by running the following command:

     kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o “<title>.*</title>"
    

    The output should be:

    <title>Simple Bookstore App</title>

  11. Update the existing DestinationRule defined for the reviews service (samples/Policyobject/networking/destination-rule-reviews.yaml) to enable all traffic intended for the reviews service to be enabled for mTLS. With the ISTIO_MUTUAL setting, clients can use the certificates created by the Citadel component for authentication:

     apiVersion: networking.istio.io/v1alpha3
     kind: DestinationRule
     metadata:
       name: reviews
     spec:
       host: reviews
       trafficPolicy:
         loadBalancer:
           simple: RANDOM
         tls:
           mode: ISTIO_MUTUAL
       subsets:
       - name: v1
         labels:
           version: v1
       - name: v2
         labels:
           version: v2
       - name: v3
         labels:
           version: v3
    

    You apply the following DestinationRule:

     kubectl apply -n default -f samples/bookinfo/networking/destination-rule-reviews.yaml
    
  12. Now try to access the reviews service of the Bookinfo app:

     kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl http://productpage:9080/productpage | sed -n '/<blockquote>/,/<\/blockquote>/{ /blockquote>/d; p }'
    

    You should see the following output:

     <p>An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!</p>
    
      <small>Reviewer1</small>
    
     <p>Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.</p>
    
     <small>Reviewer2</small>
    

You can extended this approach and apply mutual TLS to other services in the Bookinfo app by creating an authentication policy for details, ratings, and product page services. In addition, you can create a corresponding DestinationRule object for these services (see samples/bookinfo/networking/destination-rule-all-mtls.yaml).

To play around more with mutual TLS in Istio, you can follow the tasks in Authentication Policy documentation.

Summary

This tutorial discussed how mutual TLS authentication works in Istio for service-to-service authentication. To enable mutual TLS in Istio, you need to define authentication policies for services at a service-specific level, namespace level, or mesh-wide scope. An authentication policy defines what kind of traffic a service receives.

You also need to define destination rules. A DestinationRule object is an important part of Istio’s traffic routing functionality. It defines what happens to the traffic destined for a target service (of which defining the TLS settings for authentication is one part of it).

In addition to mutual TLS or transport authentication, Istio also supports origin authentication (also known as end-user authentication), where Istio authenticates the clients that make requests. Currently, Istio only supports origin authentication using JSON Web Tokens to authenticate client requests.

Try it out yourself with Istio and IBM Cloud.

Mariam John