Learn how to use IBM® App Connect on IBM Cloud™ (with a plan that provides enterprise capabilities) to expose a REST API to a MongoDB collection.

Scenario:

You want to store customer details in a database and would like to be able to invoke REST operations to create, retrieve, update, and delete customers. You’ve chosen MongoDB as the database and want to construct a REST API to allow controlled access to the customer data.

To achieve this, we’ve provided a BAR file for you to import into App Connect on IBM Cloud, to deploy an integration solution that implements a REST service to create, retrieve, update, and delete customer data in a MongoDB data source. In the integration solution, a LoopBackRequest node is used to issue synchronous requests through a LoopBack connector for MongoDB, to perform CRUD operations on a data collection in MongoDB. You’ll need to configure the LoopBackRequest node to add a data-source stanza that defines the MongoDB instance to be accessed, and to provide a LoopBack model for the data source. You’ll also need to specify security credentials that the LoopBackRequest node can use to connect to the MongoDB data source. You can define the data-source stanza, the model of the LoopBack object, and the security identity by creating policies in App Connect on IBM Cloud.

First, find or create everything you need:

  • A MongoDB instance (and cluster) that is accessible from the public internet. This tutorial uses a MongoDB Atlas instance, which is offered as a free tier for trying out MongoDB. (For reference, we’ve also provided some instructions for setting up a MongoDB instance on premises using Docker at the end of this tutorial.)
  • An IBM Cloud account with an IBM App Connect service instance that provides enterprise integration capabilities; for example, the Lite or Custom Enterprise plan.
  • Optionally, you can also install IBM App Connect Enterprise for Developers if you want to look at and change the deployed flow. It is not necessary for running the tutorial, but it is useful for examining the flow to understand how it works.
    (Note: The enterprise integration project for this tutorial was developed with the IBM App Connect Enterprise Toolkit, but can be examined with the Integration Toolkit of IBM Integration Bus V10.)
  • If you want a bit more information before you start, you can read more about the App Connect Enterprise Developer edition on the following pages:

Import the enterprise integration project into App Connect Enterprise or IBM Integration Bus, and examine the message flow:

You do not need to complete this step unless you want to look at or change the flows. A separate BAR file is also provided for deploying the integration into App Connect on IBM Cloud, as described in a later step.

  • Read more

    All the resources required for this tutorial are provided in a project interchange file named appconnect_mongodb_tutorial_pi.zip.

    The steps are the same for App Connect Enterprise V11 Toolkit and IBM Integration Bus V10 Toolkit (the toolkit):

    1. Download the project interchange file, by clicking the link above and saving the file to a local directory.
    2. Open the toolkit.
    3. To import the project into the toolkit, click File > Import, expand IBM Integration, select Project Interchange, and click Next. Then browse to select the downloaded project interchange file and click Finish.

      Importing the project into the toolkit

    A new project called CustomerMongoDBV1 is displayed in the toolkit.

    Imported project in the toolkit

    This project contains one flow called CustomerMongoDBV1.msgflow, which implements a REST service with the following resources:

    As shown in the last two images, the REST service has two main resource paths:

    • /customers for creating a customer and retrieving all current customers
    • /customers/{customerId} for retrieving, deleting, and updating individual customers

    The resource paths provide five operations that each have their own generated subflow for interacting with MongoDB. The subflow constructions are very similar and they need the same configuration to interact with MongoDB.

    Let’s look at one of these subflows in detail. The subflow named getCustomer.subflow implements the retrieval of a customer from MongoDB by using an ID as the key.

    This subflow includes the following actions:

    • Set Id: Contains a few lines of ESQL to set the ID to use for the lookup:
    • Retrieve customer: Uses the MongoDB LoopBack connector to perform the retrieval:

      • The data source name links the LoopBackRequest node to the definition entry (or data-source stanza) that needs to be created in the datasource.json file in the App Connect Enterprise work directory. This file then contains details for connecting to MongoDB; for example, the host name and port. In App Connect on IBM Cloud, the MongoDB definition entry will be created for you based on a policy that you define in the web UI. More details on this later…
      • The security identity similarly provides a link to the database user name and password that you use to access your MongoDB cluster. In App Connect Enterprise, security identities are created using the mqsisetdbparms command and, again, this step will be completed for you in App Connect on IBM Cloud based on a policy.
      • The final thing required is the model for the LoopBack object, which needs to be copied to the data source directory. App Connect on IBM Cloud will do this for you based on any models that you attach to your data source policy.
    • Generate 404 if no customer: Checks the response from the LoopBack connector and returns a 404 error if no doc is found.

    When moving the Rest API to the cloud, the important parts of the preceding subflow are the details given on the LoopBackRequest node. This will require policies to be created in App Connect on IBM Cloud to define the data source, the model of the object, and the security identity. Once these are defined, the flow can run in the cloud without any modifications to the flow itself.

Get the connection details from mongodb.com:

To allow App Connect on IBM Cloud to connect to MongoDB, you need to obtain connection details.

  1. Log in to your MongoDB instance and click the CONNECT button in the Clusters view.
  2. From the “Connect to cluster” panel, click Connect Your Application.

    Navigating to the 'Connect Your Application' option in a mongodb.com instance

  3. Click the Short SRV connection string option to display the SRV address.

    In our example, the SRV address is shown as mongodb+srv://user1:@cluster0-bjrkj.mongodb.net/test?retryWrites=true. The LoopBack connector requires only the host name portion of this address. Make a note of the host name (in our example, this is cluster0-bjrkj.mongodb.net). You’ll also need the database user name and password that you use to access your MongoDB cluster. This database user must have sufficient access to create databases and collections.

Configure the integration in App Connect on IBM Cloud:

  1. Download the CustomerMongoDBV1.bar file.
  2. Sign in to App Connect on IBM Cloud.
  3. From the App Connect on IBM Cloud dashboard, click New > Import a BAR file and select the BAR file that you downloaded. Then click Import.

    The integration server is displayed as a tile in the dashboard.

    Integration server for the imported BAR file

  4. From the App Connect menu App Connect menu icon, click Manage > Policies to open the Policies view.
  5. Click Create a policy or click New.
  6. Create a MongoDB (Loopback) policy with the following details (to configure the data source and model):
    • Policy name: Enter a name; for example, mongodb.com datasource.
    • Datasource name: Enter mongodbds. (Must be set to mongodbds to match the value given on the LoopBack node.)
    • Host: Enter the host name value from the SRV address in your MongoDB instance; for example, cluster0-bjrkj.mongodb.net.
    • Port: Accept the MongoDB default value of 27017 or specify your configured port.
    • Database name: Enter CustomersDatabase.
    • Authentication source: Accept the default value of admin or specify your preferred value for the authentication database in which the MongoDB user is created.
    • Add model: Use this boxed area to attach a model for the customers object. Download and add this definition of the model: customers.json.
    • Connect to MongoDB via the public internet: Select this check box.
    • Protocol: Select mongodb+srv.
    • Use SSL: Select this check box.

      Note: The Protocol and Use SSL fields are displayed when you select the Connect to MongoDB via the public internet check box. If you do not select this check box, App Connect will instead attempt to connect to an on-premises MongoDB system.

  7. Click Create and then return to the App Connect dashboard.
  8. Create a second policy of type Generic (Security Identity). This is needed because the LoopBackRequest node also requires a security identity.

    The policy name is unimportant (in our example, we’ve used mongodb security identity), but the security identity name must be mongodbsi to match the value given on the LoopBackRequest node. Specify the security identity type as loopback, and then specify the database user name and password that you use to access your MongoDB cluster.

  9. Attach the two polices to the integration server.
    1. From the App Connect dashboard, click the CustomerMongoDBV1 tile to open the integration.
    2. Click the Attached policies tab, and then click Manage.
    3. Select the mongodb.com datasource and mongodb security idenity policies and then save.

  10. Return to the dashboard. Then start the integration by opening the options menu [⋮] for the integration server, and then clicking Start. When the integration shows Running, the integration is running and ready to use.

    Integration server in Running state

Finally, test your integration:

Now try creating a customer and retrieving the details:

  1. From the dashboard, open the integration.
  2. Click Show API Explorer.
  3. Choose addCustomer from the five operations shown. Then click Try it.

    addCustomer operation in the API Explorer

  4. Click Generate to produce test data for a new customer.

    Generated data for the addcustomer operation

  5. Click Call operation and success should get returned.

    Successful result for the addCustomer operation

  6. Check the CustomerCollection collection in the CustomersDatabase database in MondoDB to verify that a new customer has been added.
  7. Go back to the API Explorer view in App Connect. This time, choose the getAllCustomers operation and then click Try it.
  8. Click Call operation and success should get returned with details of the customer you added.

    Successful results for the getAllCustomers operation

If all of these steps work, then you have the tutorial up and running. If it fails, then an error message with the reason will be returned. Check the integration log for any errors – you can access the log from the integration in the dashboard.

Using a Docker container running on premises rather than a mongodb.com instance:

To run MongoDB on premises, you need to have Docker running on a local machine that has App Connect Enterprise installed. Then run the following command, substituting the user name and password with your own values:

docker run -p 27017:27017 --name mongodb -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -d mongo:latest

You now have MongoDB running locally and you just need to configure App Connect on IBM Cloud to have access to your local machine.

  1. Change the mongodb.com security identity policy to use the username and password for the docker container.
  2. Change the mongodb.com datasource policy to use a host name of localhost and to not connect to the public internet.
  3. Save the policy and refresh the policy page. You should now see that the policy has a warning against it asking you to update the agent.
  4. Click the policy warning and follow the instructions given. After this is done, a green tick should appear against the policy.
  5. Stop and start the integration to pick up the new configuration.
  6. Now repeat the test from the previous section. It should work the same as before, but using your local MongoDB rather than your mongodb.com instance.

5 comments on"Using IBM App Connect enterprise capabilities to invoke REST operations on a data collection in MongoDB"

  1. When i was using mongdb altas I did not give any ip address anywhere I just used the host name provided by the mongodb webside. For me that was: cluster0-bjrkj.mongodb.net

  2. Hi Honda,
    It certainly will work using FP5 on-premises as that is exactly what we are using in App Connect on IBM cloud. It does take a little more setup as in the cloud managed service we are doing quite a lot for you including installing the mongodb connector. I do not have any instructions I have created to do that but the following article should help: https://developer.ibm.com/integration/blog/2016/09/02/interacting-with-mongodb-using-ibm-integration-bus-loopbackrequest-node.
    Regards, John.

  3. Honda Bhyat October 06, 2019

    Are there any guidance for using this example with a locally running Integration Server on ACE FP 5, especially around the setup of the Mongo Policy.

  4. Was that 0.0.0.0 to allow all IP addresses ?

  5. John, Thanks for taking time to write this detailed blog post. it’s very useful. Since you leveraged cluster in mongoDB Atlas cloud based service for this PoC, so just wanted to check, which IP address did you provide to complete the Connection Security setup ?

Join The Discussion

Your email address will not be published. Required fields are marked *