Overview

Skill Level: Intermediate

Creating custom policies require some DataPower knowledge because the core build out is within DataPower initially.

Guide to creating user-defined (custom) policies to be used in the Assemble section for APIC design time.
NOTE: This is not a tutorial, but a guide to building out custom policies.

Ingredients

  • Access to a "sandbox" based application domain on DataPower. DO NOT USE the APIC generated domain, or the default domain. Create a new domain to possibly dispose of later.
  • Privileged user access to the APIC provider organization and Catalog in which the API with the custom policy will be tested on.

Step-by-step

  1. What is a User-Defined Policy or Custom Policy

    A custom policy is created outside of IBM API Connect (management node). A policy is a piece of configuration that controls a specific aspect of processing in the IBM DataPower Gateway during the handling of an API invocation at run time. IBM API Connect provides out-of-the-box policies in the API Manager Assembly Editor (Assemble) section, but you can also create user-defined policies to provide more processing control such as security and/or routing of requests.

    The following diagram below provides an overview of the creation of a custom policy and where it resides after import.

    01.user-defined-custom-policy-process

    The following diagram below provides a sample view of the out-of-the-box policies, and the custom policy in the API Manager Assembly Editor (Assemble) section.

    02.policies-1

     

  2. Overview of a completed custom policy

    To first help the Custom Policy developer envision the whole picture, here is a visualization of the completed zip file. The location of the custom policy during development should reside on a system which has access to both DataPower and the APIC API Manager console (your local system should be ok if you can access DataPower). The goal is to upload the final packaged defined custom policy into the Catalog which APIs will be utilizing the custom policy.

     

    The following shows the hierarchy of the files required in a packaged custom policy:

    The sample used for this Custom Policy will be called Custom_Policy.zip and is ready to be uploaded to the Catalog.

    03.custom_policy.zip

    The file structure for a completed custom policy is as follows (what is in the Custom_Policy.zip):

    04.implementation

    The implementation folder contains the DataPower export which the processing logic of the transaction during runtime is utilized. This will be detailed later in the document.

    05.complete_zip

     

  3. Developing the Custom Policy Definitions and Properties (yaml file)

    This section will describe how to define the custom policy yaml file which is the description and versioning of the custom policy, and API developer properties input for the policy.

    Taking the sample from the APIC Knowledge Center: Describing your policy the diagram below showcases the yaml code to its policy Web GUI output in the API Manager Assembly Editor (Assemble).

    The sample YAML code used in this example is also located in the appendix: Sample Custom Policy YAML: Supported Primitives.

    06.yamlDefCodeMatched-1

    When defining the YAML file, the following sections are available:

    NOTICE: Be wary of the format, do not use tab indentation, the yaml markup will fail if the yaml based canonicalization is not followed (for more info, please see yaml.org).

    YAML Definition Description 
    Specification version. The YAML file policy version.
    Information section. The title, name, version, description (optional), and contact (optional) info for the policy.
    Attach section. Whether this policy may be attached to a REST API flow and/or SOAP API flow.
    Properties section. Input properties for the policy as seen in the diagram above: YAML definition code matched with WebGUI policy properties.

    The main properties value starts off defined with a type: object

    Then another proceeding properties declaration defines the input properties for the policy:

    · The root title is the property name and is used to be referenced

    o label: The name displayed for the property

    o description: description of the property

    o type: integer, number, string, boolean, or array

    o default (optional): default value

    o enum (for properties with a type of string): array of valid values

    Note: At the same level of the sub-properties definition, the required definition may be used for any required properties.

    Gateways section. Specification whether the policy is used with the DataPower gateway, or the microservices gateway.

    The diagram below shows the sections involved in the YAML file.

    07.yamlDef

     

  4. Developing the Implementation on DataPower

    The implementation consists of the DataPower export. It is created on DataPower because it will be uploaded to the inline processing of the main APIC process to be utilized by the custom policy. More information of this section may be found in the APIC Knowledge Center: Implementing you policy.

    Templates and API Gateway runtime contexts can be used to make the processing actions more effective and necessary. A library of functions and templates provides the following mechanisms:

    For example, the Custom_Policy.yaml contains a property titled samplestring, which is an input string that appears in the API Manager Assembly Editor (Assemble) Custom Policy.

    08.sampleStringProp

    This input string may be utilized in a DataPower processing rule (access to input property values), such as a DataPower Transform Action. You will see that the sample custom policy using Custom_Policy.xsl  (shown below) stylesheet to return a <response><hello>{samplestring}</hello></response> response in the API.

    <xsl:stylesheet
          version="1.0"
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
          xmlns:dp="http://www.datapower.com/extensions"
          extension-element-prefixes="dp"
          xmlns:apim="http://www.ibm.com/apimanagement"
          exclude-result-prefixes="dp json apim"
          xmlns="http://www.ibm.com/example">

          <!-- Contains the APIC functions -->
          <xsl:import href="local:///isp/policy/apim.custom.xsl" />
          <xsl:template match="/">
                <xsl:variable name="vPolicyProperties" select="apim:policyProperties()"/>
                <response>
                      <hello>
                            <!-- samplestring is a yaml property defined in the Custom Policy, which input string from the API Manager will be extract here. -->
                            <xsl:value-of select="$vPolicyProperties/samplestring"/>
                      </hello>
                </response>
                <xsl:call-template name="apim:output">
                      <xsl:with-param name="mediaType" select="'application/xml'" />
                </xsl:call-template>
          </xsl:template>

     

     

     

  5. Implementation convention requirements

    The implementation must adhere to the following conventions (the Custom Policy sample will be used to showcase the requirements when applicable):

    Recommendation: Processing Rule and Actions being created for the Custom Policy should initially be done in a sandbox-based domain on DataPower. After developing the processing rule/action and taking the export, the sandbox-based domain maybe cleaned/removed.

    1. You must create a main processing rule, and this rule will be the starting point for the policy. The rule name must start with the name of the user-defined policy (as defined in the policy YAML file), followed by –main, for example Custom_Policy-main.

    09.processingRule

    2. There are no restrictions on what actions the processing rule can execute, on condition that the rule adheres to the naming convention in point 1, and the instructions that are detailed in this topic.

    10.configureProcessingRule

    3. The names of the processing actions and all other objects also must start with the name of the user-defined policy (as defined in the policy YAML file).

    11.configureProcessingPolicy_CP

    4. If your processing rule runs transformation actions that use XSLT or GatewayScript files, these files must be stored in the following location: local://policy/policyname. Ensure to create directories and subdirectories in the File Management section.

    12.file_-1

    5. If you have certificate and key files that must be stored in the cert: folder, the names of these files also must start with the name of the user-defined policy (as defined in the policy YAML file).
    6. If your policy is using GatewayScript transformations, your policy code must require apim.custom.js, for example:

    var apic = require(‘./policy/apim.custom.js’);

    where the require can be a relative path statement (as shown in the previous example), or an absolute path statement.

     

     

  6. Building the Sample Custom Policy Implementation

    The following will walk through an exercise to build the desired Processing Rule for the sample Custom Policy. For simplicity, the sample will use the input from the property value in the API Manager Assembly Editor Custom Policy to output the following:

    <response>

          <hello>{APIC_input_property_value_here}</hello>

    </response>

     

    Processing Rule

    1. Login into the sandbox-based domain on DataPower.

    2. Navigate to the Processing Rule section.

    3. Click on Add to add a new Processing Rule.

    4. The Name should fit the APIC custom policy convention, Custom_Policy-main, ensure that the Rule Direction has Both Directions selected, and select on for the Non-XML Processing.

    5. Click Apply when complete.

    13.processingRule

    Processing Action

    6. In the Rule Action section, click on the Add new icon (+).

    7. The Name should fit the APIC custom policy convention, Custom_Policy-main_xformAction (appending content after the ‚Äďmain is ok; the _xformAction was added to identify more specifically what this action is for).

    8. Ensure the Transform with XSLT is selected since we will be using the stylesheet Custom_Policy.xsl (from previously in the Developing the Implementation on DataPower section) to retrieve the input of the custom policy property to use in a simple transform logic.

    9. Enter Input for the Input.

    10. Enter the location of where the stylesheet will be uploaded to DataPower: local://policy/Custom_Policy/Custom_Policy.xsl

    11. Enter Output for the Output.

    12. Click Apply when complete.

    File location for Processing Action

    In step 10, the file location was input to be local://policy/Custom_Policy/Custom_Policy.xsl, but the sub-directories were not created, and the stylesheet has yet to be uploaded. The following are the instructions to create the sub-directories.

    13. Navigate to the File Management section of the Web GUI on DataPower.

    14. Inside the File Management section, locate the local: folder and click on the Actions… to the right side of the local: folder.

    15. From the drop down click on Create Subdirectory and name the new directory policy.

    16. From the policy folder, create another subdirectory named Custom_Policy.

    17. Once completed, upload the Custom_Policy.xsl.

    14.fileSystem

    Exporting

    After having your Processing Rule and Processing Actions set, you’re ready to export what is required to package the APIC custom policy. To export, there are 2 methods to export. You may choose one of the following two methods to export:

    Export method 1

    18. Exporting

    a. From the Processing Rule section, click on the Export link on the upper right-hand side, and DataPower will prompt you to download the export zip. Note: Once downloaded, ensure the name of the zip is Custom_Policy.zip (the naming convention for the export must match the yaml file name).

    15.exportLink

     

    Export method 2: Use when dependencies are involved

    Export method 2 should be used when there are dependent DataPower objects which the policy uses, such as Crypto Key/Certificate objects used in possible side calls from the Processing Action to be grouped together in one export.

    b. Navigate to the main Control Panel of the DataPower WebGUI and select Export Configuration.

    c. Ensure Export configuration and files from the current domain is selected and click Next.

    d. Name the export file (per APIC custom policy convention) Custom_Policy.

    e. In the Objects section below, drop down to Processing Rule and select the rule of the custom policy desired. Here you may will also include any dependencies to the Processing Rule. For this case, there are no dependencies.

    f. Click Next and Download the export.

     

  7. Packaging the Custom Policy

    Ultimately, packaging the Custom_Policy.yaml with the implementation folder, which contains the DataPower zip inside, is the goal.

    19.  Take the DataPower export generated from Developing the Implementation on DataPower section and put it in the implementation folder.

    Note: Ensure the name of the DataPower export is the same as the yaml file: Custom_Policy.zip

    20.  Taking the yaml definition file from the section Developing the Custom Policy Definitions and Properties (yaml file) the sample Custom_Policy.yaml file (from appendix) and place the file in same root directory as the implementation folder.

    21.  Package up the Custom_Policy.yaml and implementation folder. This will be the zip to upload the APIC Catalog.

    16.files_-1

     

  8. Uploading Custom Policy and Library Usage

    Once the package is in place, choose the Catalog you would like the custom policy to be used in on APIC.

    22.  Log into the APIC Organization and into the Catalog desired to run the custom policy. The custom policy is uploaded Under the Settings > Policy of the Catalog.

    17.policyUpload

    22.  Create a sample API to use the custom policy. In the Custom_Policy policy, enter world for the samplestring. Save the API once complete.

    18.customPolicyPalette

    Use the testing tool to run the completed API and you will see that the input property value is set in the <hello>{samplestring}</hello> node.

    19.cp_palette

     

  9. Access to input property values (input from the assembly editor)

    As previously mentioned in the Developing the Implementation on DataPower section, accessing property input values are one of the features which may be utilized in the policy to accomplish additional runtime processing.

    For the example in the Custom_Policy.zip, a transform action used the input property value to output a modifed response message.

    The extension required to use the library is detailed in the APIC Knowledge Center: Implementation code examples.

    The following were used to obtain the samplestring property value:

    1. APIC extension declaration

     XSL

    xmlns:apim=”http://www.ibm.com/apimanagement”

    exclude-result-prefixes=”dp json apim”

    2. Importing/including the apim.custom library

    XSL¬† <xsl:import href=”local:///isp/policy/apim.custom.xsl” />¬†
    JS¬† var apic = require(‚Äė./policy/apim.custom.js‚Äô);

    3. Call on the apim:policyProperties() function from the library to get the input properties from the custom policy.

    XSL¬† <xsl:variable name=”vPolicyProperties” select=”apim:policyProperties()”/>
    JS  apic.getPolicyProperty(propertyName);

    Another example is shown in the APIC Knowledge Center:  Implementation code examples under Access to input properties code snippet.

    The following code block shows an example of how to access the input properties by using the XSLT policyProperties() function. The example defines a property that is named a_property, which is declared as an integer value, but is retrieved in XSLT as text.

    <?xml version="1.0" encoding="UTF-8"?>
    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      xmlns:func="http://exslt.org/functions"
      xmlns:apim="http://www.ibm.com/apimanagement" extension-element-prefixes="dp func apim">
      <!-- Contains the APIM functions -->
      <xsl:import href="local:///isp/policy/apim.custom.xsl" />
      <xsl:template match="/">
        <xsl:variable name="p" select="apim:policyProperties()" />
        <xsl:message>
          The value of my input property is
          <xsl:value-of select="$p/a_property" />
        </xsl:message>
      </xsl:template>
    </xsl:stylesheet>

    If you are using GatewayScript, you must call the function:

    apic.getPolicyProperty(propertyName)

    where propertyName is the name of the input property that you want to access. If the input property name is left blank, the action will return all input properties

  10. Access to context values (the API Gateway runtime context)

    An example is shown in the APIC Knowledge Center: Implementation code examples under Access to runtime context code snippet.

    The following code block shows an example of how to access the runtime context by using the XSLT getContext() function.

    <?xml version="1.0" encoding="UTF-8"?>
    <xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:dp="http://www.datapower.com/extensions"
    xmlns:func="http://exslt.org/functions"
    xmlns:apim="http://www.ibm.com/apimanagement" extension-element-prefixes="dp func apim">

    <!-- Contains the APIM functions -->
    <xsl:import href="local:///isp/policy/apim.custom.xsl" />

    <xsl:template match="/">
    <xsl:variable name="client-id" select="apim:getContext('client.app.id')" />
    <xsl:message>
    The calling application is
    <xsl:value-of select="$client-id" />
    </xsl:message>
    </xsl:template>

    </xsl:stylesheet>

     

    If you are using GatewayScript, you must call the function:

    apic.getContext(varName)
    where varName is the name of the context variable that you want to access.

    For a complete list of context variables, see API Gateway context variables.

     

  11. Access to the runtime payload message and media-type

    An example is shown in the APIC Knowledge Center: Implementation code examples under Access to input payload code snippet.

    The following code block shows an example of how to access the input payload by using the XSLT payloadRead() function.

    <?xml version="1.0" encoding="UTF-8"?>

    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      xmlns:func="http://exslt.org/functions"
      xmlns:apim="http://www.ibm.com/apimanagement" extension-element-prefixes="dp func apim">

      <!-- Contains the APIM functions -->
      <xsl:import href="local:///isp/policy/apim.custom.xsl" />

      <xsl:template match="/">
        <xsl:variable name="input" select="apim:payloadRead()" />
        <xsl:message>
          The input payload is
          <xsl:copy-of select="$input" />
        </xsl:message>
      </xsl:template>
    </xsl:stylesheet>

    If you are using GatewayScript, you must call the function:

    apic.readInput(callback)

    A callback is required because the actual payload read is asynchronous. The callback method is called when the payload is ready.

    This function returns an XML node-set that contains the payload of the request. If the payload is in JSON format, a JSONx node-set is returned that can then be manipulated within an XSLT or GatewayScript stylesheet. If the payload is not in JSON or XML format, the node-set that is returned is empty.

    The following example shows how to use the payloadType() function to determine what type of payload (XML or JSONx) will be returned by the XSLT payloadRead() function.

    <?xml version="1.0" encoding="UTF-8"?>

    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:dp="http://www.datapower.com/extensions"
      xmlns:func="http://exslt.org/functions"
      xmlns:apim="http://www.ibm.com/apimanagement" extension-element-prefixes="dp func apim">

      <!-- Contains the APIM functions -->
      <xsl:import href="local:///isp/policy/apim.custom.xsl" />

      <xsl:template match="/">
        <xsl:variable name="payloadType" select="apim:payloadType()" />
        <xsl:message>
          <xsl:text>Payload type is [</xsl:text>
          <xsl:value-of select="$payloadType" />
          <xsl:text>]</xsl:text>
        </xsl:message>
      </xsl:template>
    </xsl:stylesheet>

     

     

  12. Ability to modify the payload message

    An example is shown in the APIC Knowledge Center: Implementation code examples under Modify the payload code snippet.

    The following code block shows an example of how to modify the payload in XSLT. The output must be an XML node-set, which represents an XML or SOAP message, or a JSON message by using JSONx. To assist the API Gateway policy framework to accept the new or transformed message, call the apim-output template, as shown in the following example.

    <?xml version="1.0" encoding="UTF-8"?>

    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:apim="http://www.ibm.com/apimanagement"
      xmlns:jsonx="http://www.ibm.com/xmlns/prod/2009/jsonx">

      <!-- Contains the APIM functions -->
      <xsl:include href="local:///isp/policy/apim.custom.xsl" />

      <xsl:template match="/">
        <!-- Creates a JSON document (empty object is for simplicity) -->
        <jsonx:object>
        </jsonx:object>

        <!-- Indicates the media type of the output being produced -->
        <xsl:call-template name="apim:output">
          <xsl:with-param name="mediaType" select="'application/json'" />
        </xsl:call-template>
      </xsl:template>
    </xsl:stylesheet>

    where mediaType:

    • ‘application/json’¬†is when the output is written in JSONx format.
    • ‘application/xml’¬†is when the output is written in XML format.

    If you are using GatewayScript, you must call the function:

    apic.output(mediaType)

    where mediaType is:

    • application/json¬†is when the output is written in JSONx format.
    • application/xml¬†is when the output is written in XML format.

    Specifying the media type allows the next steps in the assembly flow to understand how to process the new payload.

     

  13. Ability to end the policy execution with an error

    An example is shown in the APIC Knowledge Center: Implementation code examples under Configuration error information code snippet.

    The following XSLT code block shows an example of how to configure the policy implementation to produce error information by calling the apim-error template.

    <?xml version="1.0" encoding="UTF-8"?>

    <xsl:stylesheet version="1.0"
      xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
      xmlns:apim="http://www.ibm.com/apimanagement">

      <!-- Contains the APIM functions -->
      <xsl:include href="local:///isp/policy/apim.custom.xsl" />

      <!-- Indicates this policy has a failure and provides
           additional information for the client application -->
      <xsl:template match="/">
        <xsl:call-template name="apim:error">
          <xsl:with-param name="httpCode" select="'401'" />
          <xsl:with-param name="httpReasonPhrase" select="'Unauthorized'" />
          <xsl:with-param name="errorMessage" select="'Please select a Plan'" />
        </xsl:call-template>
      </xsl:template>
    </xsl:stylesheet>

    where:

    • httpCode¬†is the code of the required error message.
    • httpReasonPhrase¬†is the reason for the error.
    • errorMessage¬†is the suggested action for the user.

    If you are using GatewayScript, you must call the template:

    apic.error(name, httpCode, httpReasonPhrase, message)

    where:

    • name¬†is the name of the error.
    • httpCode¬†is the code of the required error message.
    • httpReasonPhrase¬†is the reason for the error.
    • message¬†is the suggested action for the user.
  14. Debug Custom Policy Upload Errors

    The /var/log/cmc.out log contains the errors from Custom Policy uploads. By reviewing this rolling log you will need access to the CLI console of your APIC management node.

    By issuing debug tail file /var/log/cmc.out the cmc.out logs will output the logs as the upload of the Custom Policy is being uploaded:

    20.putty_

     

  15. Appendix: Sample Custom Policy YAML: Supported Primitives

    https://www.ibm.com/support/knowledgecenter/SSMNED_5.0.0/com.ibm.apic.policy.doc/tapim_creating_policy.html?lang=en

    policy: 1.0.0

    info:
      title: Sample Policy
      name: sampleimpl
      version: 1.0.1
      description: This is a sample policy.
      contact:
        name: Steve Product Manager
        url: http://developer.acme.com/contacturl
        email: steve-product-manager@someemailservice.com

    attach:
      - rest
      - soap

    properties:
      $schema: "http://json-schema.org/draft-04/schema#"
      type: object
      properties:
        samplestring:
          label: String Property
          description: Sample string property
          type: string
        sampleboolean:
          label: Boolean Property
          description: Sample boolean property
          type: boolean
        sampleinteger:
          label: Integer Property
          description: Sample integer property
          type: integer
        samplenumber:
          label: Number Property
          description: Sample number property
          type: number
        samplestringwithenum:
          label: Enum Property
          description: Sample string enum property
          enum:
            - one
            - two
            - three
          default: one
          type: string
        samplearray:
          label: Array of properties
          description: Sample array of properties
          type: array
          items:
            type: object
            properties:
              string:
                label: String Property
                description: Sample string property in array
                type: string
              boolean:
                label: Boolean Property
                description: Sample boolean property in array
                type: boolean
            required:
              - string
      required:
        - samplestring
        - sampleboolean
        - sampleinteger
        - samplenumber
        - samplearray

    gateways:
      - datapower-gateway

     

  16. Appendix: Sample Custom Policy: Custom_Policy.xsl

    <xsl:stylesheet
          version="1.0"
          xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
          xmlns:dp="http://www.datapower.com/extensions"
          extension-element-prefixes="dp"
          xmlns:apim="http://www.ibm.com/apimanagement"
          exclude-result-prefixes="dp json apim"
          xmlns="http://www.ibm.com/example">

          <!-- Contains the APIC functions -->
          <xsl:import href="local:///isp/policy/apim.custom.xsl" />

          <xsl:template match="/">
                <xsl:variable name="vPolicyProperties" select="apim:policyProperties()"/>
                <response>
                      <hello>
                            <!-- samplestring is a yaml property defined in the Custom Policy, which input string from the API Manager will be extract here. -->
                            <xsl:value-of select="$vPolicyProperties/samplestring"/>
                      </hello>
                </response>
                <xsl:call-template name="apim:output">
                      <xsl:with-param name="mediaType" select="'application/xml'" />
                </xsl:call-template>
          </xsl:template>
    </xsl:stylesheet>

Join The Discussion