Replicate an API Connect Catalog using an IBM Cloud Delivery Pipeline
Discover a new way to replicate a catalog without deploying an API toolkit and downloading APIs and products
The IBM Cloud API Connect service is used to quickly create APIs and microservices based on Node.js and Java runtimes. After they are created, you can manage your APIs with business-level controls by setting varying levels of security, visibility, and rate limits while sharing APIs with application developers. These varying levels of settings are achieved using “Products”, a collection of APIs.
Products must be staged to a Catalog and then published to developer organizations to become available to application developers. In IBM API Connect for IBM Cloud you can create multiple “Catalogs”. Catalogs are useful for separating Products and APIs for testing before you make them available to developer organizations. When you create a new Catalog, you might want to have the APIs and Products from an already existing Catalog. This how-to guide explains how to achieve this objective without downloading the APIs and Products to a local file system or without installing API toolkit to a local machine.
This guide uses IBM Cloud’s Toolchain service to create a Delivery Pipeline. We will create a stage in the Delivery Pipeline which will run a script that will create the new a Catalog and publish Products for us.
After completing this how how-to the reader will be able to replicate an existing API Connect Catalog using the IBM Cloud Toolchain service. There are instances where a developer may need to create new Catalogs from existing ones, one example is so they can have different environments (Dev, Staging, QA, etc). There are other ways of replicating a Catalog but they require either installation of the API Toolkit on a local machine, or exporting the APIs and Products to a local machine and re-importing them. This how-to shows a new way of replicating a Catalog without deploying an API toolkit and downloading APIs and Products.
- An IBM Cloud account – sign up if you don’t have an account yet.
- A provisioned API Connect service.
- A provisioned Continuous Delivery service.
- Some familiarity with IBM Cloud’s API Connect service and IBM Cloud’s Toolchain service will help.
- Setting up a catalog takes approximately 15 minutes.
- Creating a Continuous Delivery service takes approximately 20 minutes.
- Creating a build script and running it in the pipeline takes approximately 20 minutes.
Create a sample API Connect Catalog with APIs and Products
Note: This portion of the how-to will create a Catalog with APIs and Products. If an API Connect Catalog already exists that you intend to replicate then proceed to the next section, otherwise continue with the steps below.
- Launch the API Connect service. Refer to the Prerequisites for a link to create the service.
- The default Catalog named Sandbox is listed. For the sake of this how-to, let us import a sample API and Product.
- Click the >> icon on top left corner and click Drafts.
- Click APIs tab on the top left side of the screen.
- Click Add + button.
- Click Import a sample OpenAPI.
- The default Climbing Weather 1.0.0 entry should be selected. If not, go ahead and select it.
- Click Import. The Climbing Weather API should be imported.
- Click All APIs on the top left corner.
- Click Products placed beside APIs.
- Click Add + and then click New Product.
- In the Title field, type a title for the Product. For example, sampleproduct1.
- Click Create product. A Product should be created.
Make note of the Catalog identifier
- Launch the API Connect service
- If the dashboard view of API Connect is not shown, then click >> icon on top left side of the screen and select Dashboard
- Notice the link icon on the Catalog which you want to replicate. Click on the link icon. It will show the Catalog identifier details in a text box. Make a note of this value as we will be using it later. An example Catalog identifier is shown below:
apic config:set Catalog=apic-Catalog://us.apiconnect.ibmcloud.com/orgs/muralidharchavaninibmcom-dev/Catalogs/sb
You can extract the following information from the identifier:
Now that we’ve created the API Connect Catalog we want create, and identified the necessary information, we can proceed with configuring our Continuous Delivery service.
Create a Continuous Delivery Pipeline
- Launch the Continuous Delivery service. Refer to the Prerequisites for a link to create the service.
- Under the Start from a toolchain template option, click Start here.
- Scroll to the bottom and select the Build your own toolchain template.
- Name the toolchain with the Toolchain Name: option, select a Region and Organization, and click Create.
- In the newly created toolchain’s home page click the Add a Tool button.
- In the search field type “Delivery Pipeline”.
- Click on the Delivery Pipeline option that is filtered under the Tool Integrations group.
- Enter a Pipeline name in the Pipeline name: text field and click Create Integration.
- Click on the tile of the newly created Devliery Pipeline to start customizing it.
Configure the Continuous Delivery Pipeline to replicate API Connect Catalogs
Now that we’ve created our Devliery Pipeline we’ll now configure the newly created Continuous Delivery Pipeline by using a bash script that leverages the API Connect NPM package. First we’ll describe what information is needed, then show a template bash script and example bash script, and finally, walk through how configuring the Delivery Pipeline.
Build script to copy the API Connect Catalogs
The following information is needed for build script:
username: Your IBM Cloud login username.
apiserver: This was noted in the previous section.
passcode: You can generate a passcode here. The passcode expires will expire in 5 minutes, so be sure to get a new one before running the script.
source-catalog-name: This was noted in the previous section.
organization-space-name: This was noted down in the previous section.
productname-in-catalog: The Product name in the Catalog that you want to migrate. You can get it by going to the API Connect console -> Draft View -> Products. For example,
target-catalog-name: If you don’t already have a target Catalog and want to create one, then add a new Catalog name. For example,
product-yaml-file: When the Product is pulled, it can be accessed as
<Product name>_product_<version no>.yaml. For example,
Below is a template for the bash script we’ll be using. Replace the values in angle brackets with the values above.
npm install -g apiconnect NPM_ROOT='npm root -g' APIC='which apic' apic --accept-license apic --disable-analytics # The apiserver can be found from the Catalog identifier described earlier. E.g: us.apiconnect.ibmcloud.com apic login --sso --username <username> --server <apiserver> --passcode <passcode> apic products:pull --Catalog <source-catalog-name> --organization <organization-space-name> --server <apiserver> <productname-in-catalog> # Comment the below line if the target Catalog already exists apic Catalogs:create --organization <organization-space-name> -s <apiserver> <target-catalog-name> # Publish to new space and Catalog. The Catalog should be existing else create it. apic publish <product-yaml-file> --Catalog <target-catalog-name> --organization <organization-space-name> --server <apiserver>
Below is a completed sample with the values filled in.
npm install -g apiconnect NPM_ROOT='npm root -g' APIC='which apic' apic --accept-license apic --disable-analytics # The apiserver can be found from the Catalog identifier described earlier. E.g: us.apiconnect.ibmcloud.com apic login --sso --username firstname.lastname@example.org --server us.apiconnect.ibmcloud.com --passcode abcd1234 apic products:pull --Catalog sb --organization muralidharchavaninibmcom-dev --server us.apiconnect.ibmcloud.com sampleproduct1 # Comment the below line if the target Catalog already exists apic Catalogs:create --organization muralidharchavaninibmcom-dev -s us.apiconnect.ibmcloud.com targetCatalog # Publish to new space and Catalog. The Catalog should be existing else create it. apic publish sampleproduct1_product_1.0.0.yaml --Catalog targetCatalog --organization muralidharchavaninibmcom-dev --server us.apiconnect.ibmcloud.com
Use the template script and replace all the parameters specific to your environment. Have this script ready before proceeding to the next step.
Create a Build Stage in the Continuous Delivery Pipeline
- Back in the Continuous Delivery Pipeline tool, click on Add Stage +.
- Under Input Type dropdown, select None.
- Click the JOBS tab.
- Click Add Job. Select Build under Select Job Type.
- Under Builder Type dropdown select the npm entry.
- Uncomment the
- Delete the
npm installline (it’ll be the last line).
- Now append the default build script with the script that you prepared in the previous step.
- Be sure to update the passcode in the script or generate a new one.
- Click Save.
- Once saved, click on the run icon of the stage you just created.
- The script will now run and take a couple of minutes to complete, when done the stage should have run successfully.
Verify target Catalog creation/updation
- Launch the API Connect service.
- In the API Connect service dashboard you will now be able to see the target Catalog created.
- Click on the target Catalog and verify that the Product is replicated from the source Catalog and is in a published state.
- You can repeat the steps from the previous section for every Product that you want to replicate. Alternatively you can add commands in the same script to copy and publish all products.
In this how-to guide, we have learned how to:
- Create API Connect and Delivery Pipeline services.
- Create a sample API Connect Catalog with APIs and Products.
- Create a script that replicates Products and APIs from one Catalog to another Catalog using Continuous Delivery Pipelines.
The reader has learned how to:
apiccommands in the Delivery Pipeline build stage.
- Replicate Products and APIs from one API Connect Catalog to another.