Overview

Skill Level: Intermediate

This tutorial assumes that a user has prior knowledge on Virtual Private Cloud (VPC), Cloud Object Storage(COS) and terraform.

In this article, I will show you how to provision VPC Flow Logs using terraform in IBM Cloud.

Ingredients

The article assumes that the

1. user already have an account in IBM Cloud.
2. user has created an api key in his IBM Cloud account.
3. user has permission and quota to provision VPC resources.
4. user has installed terraform in his local system.

At the end of this article, the reader will get to know the steps to the provision the below resources in IBM Cloud services using Terraform:

(i) Cloud Object Storage
(ii) VPC Flow Logs

About terraform:

Terraform is 'infrastructure as code' automation software by HashiCorp. It allows users to define cloud infrastructure in a high-level scripting language, which can be executed to build the infrastructure on the desired cloud platform.

For details on how to install and configure terraform, please refer the guide.

For more details on terraform and IBM Cloud, refer this link.

To try and test the below example, ensure that you have IBM Cloud Terraform Provider greater than v1.11.2.

About VPC Flow Logs:

IBM Cloud Flow Logs for VPC enables the collection, storage, and presentation of information about the Internet Protocol (IP) traffic going to and from network interfaces within your Virtual Private Cloud (VPC). For more details on Flow logs, please refer this link.

There are 5 REST APIs for Flow logs:

First 4 apis are used in terraform resource block. 

1. Create a Flow log entry with attributes (POST method)

active - enabled or disabled, value true/false (if not specified, true is used)
target - target can be VPC or Virtual Server Instance or Subnet or Primary Network Interface or Secondary Network Interface 
storage bucket - storage bucket name which is unique in IBM Cloud
resource group (if not specified, default resource group is used)

2. Update a Flow log (PATCH method), following attributes can be updated.

active
name of flow log

3. Get a Flow log (GET method) with attribute

id - The flow log collector identifier

4. Delete a Flow logs (DELETE method) with attribute

id - The flow log collector identifier

5. List all Flow logs (GET method). This api is used by terraform data block.


You can find the list of REST APIs here.

Step-by-step

  1. Initialize the terraform provider with your api key and cloud provider

    provider “ibm” {
    ibmcloud_api_key = “xxx-xxx”
    generation = 2
    region = “us-south”
    ibmcloud_timeout = 300
    }

  2. Fetch the required resources from cloud using data block

    // Fetch VPC using name
    data ibm_is_vpc my_vpc {
    name = “xxx-vpc”
    }

    // Fetch subnet using identifier
    data ibm_is_subnet my_subnet {
    identifier = “02subnet-0a7”
    }

    //fetch resource group
    data “ibm_resource_group” “rs_group” {
    name = “developer ibm”
    }

    //fetch virtual server instance
    data “ibm_is_instance” “ds_instance” {
    name = “my-vsi”
    }

    Output Primary Network interface:

    output “instance_prim_output” {
    value = data.ibm_is_instance.ds_instance.primary_network_interface[0].id
    }

    Output Secondary Network interface:

    output “instance_sec_output” {
    value = data.ibm_is_instance.ds_instance.network_interfaces[0].id
    }

     

  3. Create Cloud Object Storage (COS)

    //Create a Cloud Object Storage instance
    resource “ibm_resource_instance” “instance1” {
    name = “cos-instance”
    resource_group_id = data.ibm_resource_group.rs_group.id
    service = “cloud-object-storage”
    plan = “standard”
    location = “global”
    }

    //Create a bucket in “us-south” region
    resource “ibm_cos_bucket” “bucket1” {
    bucket_name = “us-south-bucket-malar”
    resource_instance_id = ibm_resource_instance.instance1.id
    region_location = “us-south”
    storage_class = “standard”
    }

    //Create an iam authorization policy and permit flow log collector to access COS service with Writer role
    resource “ibm_iam_authorization_policy” “policy” {
    source_service_name = “is”
    source_resource_type = “flow-log-collector”
    target_service_name = “cloud-object-storage”
    roles = [“Writer”] }

  4. Create VPC Flow Logs resource for desired target

    resource ibm_is_flow_log test_flowlog {
    depends_on = [ibm_cos_bucket.bucket1, ibm_iam_authorization_policy.policy] name = “test-instance-flow-log1”
    active = true
    //target can be VPC or Virtual Server Instance or Subnet or Primary Network Interface or Secondary Network Interface 
    target = data.ibm_is_instance.ds_instance.id
    resource_group = data.ibm_resource_group.rs_group.id
    storage_bucket = ibm_cos_bucket.bucket1.bucket_name
    tags = [“flowlogtest”, “malartest”]}

    The above resource block for flow logs is capturing the traffic for Virtual Server Instance (VSI). Here, the target id is of VSI.  The target can take the id of VPC or Virtual Server Instance or Subnet or Primary Network Interface or Secondary Network Interface.

    //Fetch list of flow logs
    data ibm_is_flow_logs test_flow_logs {}

    //Output list of flow logs
    output “instance_output” {
    value = data.ibm_is_flow_logs.test_flow_logs
    }

  5. Run terraform

    Initialize the terraform provider:
    terraform init

    Apply the terraform plan:
    terraform apply

  6. Check the resources in IBM Cloud

    Login to IBM Cloud account where the api key is used. Navigate to VPC Gen 2. Under Network, go to Flow logs and check whether the resources are created.

    Also, navigate to Cloud Object Storage service and check whether the COS bucket is created. Inside the bucket you should folder “ibm_vpc_flowlogs_v1” created for Flow logs, when the flow logs attribute active is true.The flow logs folder has  a subfolder account id/region/vpc/instance id, etc. Here is a sample VPC Flow logs collected inside a COS bucket. You can download the .gz file to view the Internet Protocol (IP) traffic going to and from network interfaces within your Virtual Private Cloud (VPC).

     

    IBM_COS_Bucket_object_sample_VPC_Flow_Logs

     

    This confirms that VPC Flow logs are working fine.  

  7. Destroy created resources in IBM Cloud

    Delete the VPC Flow logs folder under COS bucket. The COS bucket folder has to be empty before destroying the VPC Flow log resources.

    Run the command to destroy the resources:
    terraform destroy

Join The Discussion