Learn how to create a flow in IBM® App Connect Enterprise (ACE) that updates on-premises enterprise data and calls an event-driven flow in IBM App Connect on IBM Cloud™ to pass enterprise data to SaaS applications and to get data from SaaS applications for processing on-premises.

This is an example of enterprise IT activities, through ACE flows, exchanging and augmenting data with SaaS applications by calling a flow in App Connect on IBM Cloud to perform some of the large range of actions for SaaS applications that are available. For example, this enables enterprise IT practitioners to augment data in SaaS applications used by line of business (LoB) users and developers, and to retrieve data from SaaS applications for use in enterprise IT activities.

Scenario

Your company is starting to use Salesforce for its Sales reps to record customer leads that they find. A REST API is still used to add leads to an on-premises application, but needs to sync leads with Salesforce. Integrating these two systems requires a hybrid integration approach where some of the integration function runs in the cloud and some must run on premises:

  1. When a new lead is to be added to the on-premises application (by an ACE flow), the lead details and generated local lead ID are passed to a flow running in App Connect on IBM Cloud.
  2. The flow on IBM Cloud updates or creates a Salesforce lead, then sends an email to notify a Sales rep, and finally returns the Salesforce lead ID to the ACE flow to complete the on-premises processing of the lead.

This tutorial demonstrates a REST API used to add a new lead to an on-premises application by calling a message flow running in ACE software. The Add lead operation of the REST API passes message JSON data like the following example:

{
"FirstName" : "John",
"LastName" : "Doe",
"Email" : "j.doe@email.com",
"Company": "Adeer.com"
}

The ACE flow does some on-premises processing of the data and invokes (calls) an event-driven flow in App Connect on IBM Cloud to perform on-cloud processing to act on the SaaS applications then return data back to the ACE flow. Salesforce and GMail were used as readily-available SaaS applications for this example, but for the actions in the event-driven flow you could use other applications from the App Connect on IBM Cloud catalog.

The ACE flow then transforms the message (the original lead data and the Salesforce lead ID) to an XML format, adds the generated local lead ID, and writes the data to an XML file ready for an on-premises application to process. This shows simple file processing but could make use of any of the array of transports and formats that App Connect Enterprise supports.

First, find or create everything you need:

  • A Salesforce business or Developer Edition account.
  • A GMail account.
  • An IBM Cloud account with IBM App Connect service – Custom Enterprise, Cloud Connector, or Lite plans.
  • IBM App Connect Enterprise (ACE) on-premises installation, configured with an integration server and its work directory.

If you have not bought IBM App Connect Enterprise (ACE), you can develop and test integration solutions without charge by using the following free options:

If you want a bit more information before you start, you can read more about the ACE Developer edition and how to use App Connect with Salesforce on the following pages:

Next, complete the following steps: (Click section title to expand)

  • Configure App Connect on IBM Cloud for ACE callable flows

    In this section we use App Connect Designer to configure callable flows, to enable secure connectivity between flows running in IBM App Connect on IBM Cloud and in an integration server in your on-premises App Connect Enterprise.

    If you have already configured callable flows between App Connect on IBM Cloud and the integration server in your on-premises App Connect Enterprise, you can skip this step.

    1. Click the hamburger menu (top left)
    2. Expand Manage
    3. Select Callable flows (If you don’t see this option, you need to select an Enterprise plan for App Connect on IBM Cloud.)

    Next, click Connect Callable Flows.

    This opens the “Set up an agent” dialog.

    Click Download the configuration, this opens a dialog to save the agentx.json file.

    Leave the “Set up an agent” dialog open (note the dialog contains instructions for installing the agent into the work directory of the integration server in App Connect Enterprise).

    After saving the agentx.json. Copy it to the work directory of the integration server in App Connect Enterprise: workdirectory/config/iibswitch/agentx

    Now on your on-premises ACE software, start the integration server; for example:

    IntegrationServer --name myIntegrationServer --work-dir c:\mywrk\myaceworkdir

    You should see the ACE server start, and the log indicate ...The integration server component 'agentx' has been started:

    Switch back to the App Connect Designer window and then test the agent connection:

    Click Test your Agent

    APPC-CallableFlow2

    You should see at least 1 agent found

    You can now close the “Set up an agent” dialog.


  • Import the enterprise integration project into the toolkit, then deploy the ACE flow provided

    In this section we use the App Connect Enterprise Toolkit (Enterprise Toolkit) to create and deploy a flow into an on-premises ACE integration server. The ACE flow implements a REST API and uses a CallableFlowInvoke node to call the event-driven flow that is running in App Connect on IBM Cloud.


    (Click image to view full size.)

    Rather than create the integration project from scratch, we import the project interchange file used to develop this tutorial.

    All the resources required for this tutorial are provided in a project interchange file.

    1. Download the project interchange file by clicking the link above and saving the file to your local machine.
    2. Open the toolkit.
    3. Import the project into the toolkit. Click File>Import, select IBM Integration > Project Interchange, then select the downloaded file.

      A new project called LeadXML2ACoIC will appear in the Application Development view of the toolkit. It contains the definition for the REST API, with a subflow for the Add lead operation, called addlead.subflow:


      (Click image to view full size.)

      Ref: Creating a REST API from scratch by using the IBM App Connect Enterprise Toolkit.

      You can open the REST API Description in the REST API Editor, to display and modify information about the REST API; for example:

      • In the Header section, the REST API base URL field displays the current base path for the REST API. All resources in a REST API are defined relative to its base path. You can edit this field to modify the base path of the REST API. You can also see the URL to use to access the operations in the REST API.
      • In the Resources section, you can see the POST addLead operation for the Lead resource type (/leads). This operation is invoked by POST http://:/leadsxml/v2/leads (a combined URL from what you can see in the Header and Resources operation sections).
      • In the Model definitions section, you can see the definition for the JSON data that you can use for the Request and Response Schema type in the addLead operation.


      (Click image to view full size.)

      The subflow performs the on-premises activities for the Add lead operation, and calls the flow in App Connect on IBM Cloud:

      • Input accepts input for the subflow, passing the JSON data values for FirstName, LastName, Email, and Company; for example: InputRoot.JSON.Data.FirstName.
      • Assign Local Id creates a unique identifier for the new lead, and assigns it to the local environment variable ‘Id’: OutputLocalEnvironment.Variables.Id.
      • Call flowin AConIC calls the flow in App Connect on IBM Cloud with the flow name Call to update-create SF lead, passing the local environment values and the input JSON data values. When the called flow has finished it returns the ID of the Salesforce lead as the local environment value SfLeadId.
      • Construct XML Message converts the incoming data into an XML format, including the addition of the unique identifier (Id) and the Salesforce lead ID (SfLeadId).
      • Write to file appends the XML data directly to a file on the file system, using a new line for each new entry. The directory and filename are configured on this node.
      • Output sends a response to the caller of the REST API.
    4. Check that the Enterprise Toolkit is connected to your ACE integration server (started in the step “Configure App Connect on IBM Cloud for ACE callable flows”). If it is connected, it is shown in the Integration Explorer view:

      • Tip: If the integration server is not shown as connected, complete these steps to connect the server…
        1. In the Integration Explorer view, right-click Integration servers, and then click Connect to an integration server. In the Connection details dialog, enter the host name and port for the integration server. Ensure that the port matches the value of the adminRestApiPort property that was specified in the server.conf.yaml file (in this example, the port is 7600, and the host name is localhost). If the integration server is secured, you must also specify the user name and password.
        2. Click Finish. The connection for your integration server is now displayed in the Integration Explorer view in the toolkit.
    5. The project contains a BAR file called LeadXML2ACoIC.bar, which contains the callable flow, ready to be deployed. Deploy the BAR file to the integration server you intend to use for the tutorial; for example, drag the BAR file and then drop it onto the integration server.

      (In addition to the callable flow, the project includes the YAML configuration file for a flow exported from App Connect on IBM Cloud, called: Call to update-create SF lead.yaml. This file will be used later to create a flow in App Connect on IBM Cloud.)

      In the ACE log, you should see the flow deployed successfully and listening for the REST API:

      2018-06-11 13:01:01.827403: The connection agent for remote callable flows has established a connection to the Switch server with URL 'wss://ibm-sw-*****************.eu-gb.ace.ibm.com:443/'.
      ...
      2018-06-11 13:25:56.083516: About to 'Start' the deployed resource 'LeadXML2ACoIC' of type 'RestAPI'.
      2018-06-11 13:25:56.089092: Listening on HTTP URL '/leadsxml/v2*'.
      Started native listener for HTTP input node on port 7800 for URL /leadsxml/v2*
      

      Note the port number and URL that are part of the URL used to call the REST API; for example: http://localhost:7800/leadsxml/v2/leads


  • Configure and start the callable flow in App Connect on IBM Cloud

    In this section we configure the event-driven flow in App Connect on IBM Cloud (without any coding). This flow is to be triggered when invoked from the ACE flow. The flow then updates or creates a Salesforce lead, then sends an email to notify a Sales rep, and finally returns the Salesforce lead ID to the ACE flow to complete the on-premises processing of the lead.


    (Click image to view full size.)

    If you do not want to create the callable flow from scratch, In App Connect Designer you can import the sample flow definition file, Call to update-create SF lead.yaml, provided in the ACE project interchange file. (Tip: After importing the yaml file into App Connect on IBM Cloud, click the action nodes to check the configuration and if needed connect to your own application accounts. Also, for the GMail / Send message action specify a valid email address to send an email to.)

    1. Configure the flow to be triggered as a callable flow.

      In App Connect Designer, click the Dashboard link, then select New >Event-driven flow.

      This opens the Flow editor, and prompts you for how to start the flow.


      (Click image to view full size.)

      Click the Toolbox tab, and then select ‘Callable flow’ as the starting point. This adds a ‘Callable flow / Input’ node and a ‘Callable flow / Reply’ node to the flow.

    2. Add an action to update or create a Salesforce lead.
      1. In the flow diagram, select the (+) after the ‘Callable flow / Input’ node.
      2. In the Applictions list expand Salesforce > Leads, and then select Update or create Lead as the Salesforce action.

        If the desired Salesforce Lead already exists it will be updated; otherwise, a new Salesforce Lead will be created.

      3. If you haven’t already connected a Salesforce account, click Connect to Salesforce and follow the instructions to allow App Connect to connect to your Salesforce account.
      4. Define the conditions for Salesforce to update an existing Lead.

        Under “*Where”, add enough conditions to uniquely identify one Salesforce Lead.

        If there’s more than one record in your target system that matches the conditions when you run the flow, you’ll see an error for the flow on the dashboard, and the flow won’t update or create any records. For example, maybe you have more than one lead with the same first and last names. So to match a lead, use unique data, such as their email address or a combination of conditions.

        In this case, the condition values are taken from the message JSON data passed to the callable flow input. For example, I added conditions for FirstName, LastName, Email, and Company.

        Add each condition value as follows (or copy the values from below):

        1. Select Callable flow / Input … Message.JSON.Data (for example, using the Insert a reference icon ).
        2. Click Message.JSON.Data and then select the Edit expression. Add the data value name from the message JSON data.

        This should give the following values:

        Where - equals value
        First Name    - {{$Trigger.message.FirstName}}
        Last Name     - {{$Trigger.message.LastName}}
        Email         - {{$Trigger.message.Email}}
        Company       - {{$Trigger.message.Company}}
        
      5. Populate the target fields in Salesforce.

        App Connect automatically completes the mapping to target fields specified in the Where conditions, showing each such field with a message that indicates the value used in the Where condition will be used.


        (Click image to view full size.)

        For any other fields that you want to update or create for the Salesforce Lead, click the Insert a reference icon , then select and edit from Callable flow / Input … :

        For example, the Description field is used to record the generated local lead ID from the local environment tree Id variable. For this field, create the mapping by selecting Callable flow / Input … LocalEnvironment.Variables, and then editing the expression as shown:

        On-prem Lead ID = {{$Trigger.localEnvironment.Id}}
    3. Add an action for Gmail to send an email to notify a Sales rep:
      1. In the flow diagram, select the (+) after the Salesforce node.
      2. In the Applications list expand Gmail > Messages, select ‘Create email’, and then populate the email fields to use; for example to send values from the message data and Saleforce lead in the email body:

        Body - FirstName = {{$Trigger.message.FirstName}}
               LastName = {{$Trigger.message.LastName}}
               Salesforce Lead ID = {{$SalesforceUpdateorcreatelead.Id}}
               On-prem Lead ID = {{$Trigger.localEnvironment.Id}}
        
    4. Configure the data that is to be returned to the ACE flow:
      1. In the flow diagram, click the ‘Callable flow / Reply’ node.
      2. Under ‘LocalEnvironment.Variables’, click the ‘Add property’ link, then give it the name SfLeadId (leave the type as String).
      3. Click the ‘Edit mappings’ link, then for the SfLeadId value map Salesforce / Update or create lead > Lead > Lead ID
      4. Under ‘Message.JSON.Data’, use the ‘Add property’ link to add the following properties (leave the type as String):
        FirstName    - {{$Trigger.message.FirstName}}    (or Salesforce / Lead > First Name)
        LastName     - {{$Trigger.message.LastName}}     (or Salesforce / Lead > Last Name)
        Email         - {{$Trigger.message.Email}}        (or Salesforce / Lead > Email)
        Company       - {{$Trigger.message.Company}}
        

        Click the ‘Edit mappings’ link, then for each property map the value from the Message.JSON.Data (and edit the expression as before) or from the Salesforce / Update or create lead > Lead.

    5. Give your flow the name: Call to update-create SF lead

      The flow name needs to match the application name that the ACE flow uses to invoke this callable flow.

    6. To exit and start the flow, open the options menu [⋮] in the banner and click Start flow; then click Dashboard.

      Otherwise, just return to the Dashboard, You can start the flow from there, as outlined in the Test the flow step.

    7. Switch to the “Callable flows” view (using the hamburger menu on the left of the window); you should now see the new callable flow provided by App Connect on IBM Cloud. Note the Application name should match the name specified when creating the flow.

      Tip: If you do not see the new callable flow in the “Callable flows” list, refresh the browser window to update the list.


  • Test the integration solution

    In this section we test the integration solution, by calling the REST API to add a new lead to the on-premises application and to update Salesforce. The REST API invokes the ACE flow to interact with the on-premises application and to call the flow in App Connect on IBM Cloud to interact with Salesforce.

    Test the integration solution by calling the REST API with data for a new lead using an HTTP client such as cURL or Postman. For example. using the cURL command:

    curl -X POST http://localhost:7800/leadsxml/v2/leads -d '{"FirstName" : "John","LastName" : "Smith","Email" : "j.smith@email.com","Company": "jsmith.com"}'

    The transformed response message should look like this (with the generated id and Salesforce sfid values):

    In a command window:

    <lead><name><first>John</first><last>Smith</last></name><emailAddress>j.smith@email.com</emailAddress><company>jsmith.com</company><id>Smith08cd492b-b11b-466a-85b9-324d58875f15</id><sfid>00Q5800000XZBKHEA5</sfid></lead>

    Or formatted, as in a result from Postman:

    <lead>
        <name>
            <first>John</first>
            <last>Smith</last>
        </name>
        <emailAddress>j.smith@email.com</emailAddress>
        <company>jsmith.com</company>
        <id>Smith08cd492b-b11b-466a-85b9-324d58875f15</id>
        <sfid>00Q5800000XZBKHEA5</sfid>
    </lead>

    If you view the on-premises application (C:\temp\leadsxml.txt) you should see the new lead entry appended:

    <lead><name><first>John</first><last>Smith</last></name><emailAddress>j.smith@email.com</emailAddress><company>jsmith.com</company><id>Smith08cd492b-b11b-466a-85b9-324d58875f15</id><sfid>00Q5800000XZBKHEA5</sfid></lead>

    In Salesforce, you should also see the lead created or updated by the flow in App Connect on IBM Cloud, with the description value set with data from the ACE flow; for example:

    Description
    On-prem Lead ID = Smith08cd492b-b11b-466a-85b9-324d58875f15

    Show image (as overlay)

    In the inbox of the email account you chose to use in the flow in App Connect on IBM Cloud, you should see a message about the new lead; for example:

    Date: Mon, Jun 11, 2018 at 3:34 PM
    Subject: Salesforce lead updated / created from on-prem lead
    To: myaddr@gmail.com
    
    FirstName = John
    LastName = Smith
    Salesforce Lead  ID = 00Q5800000XZBKHEA5
    On-prem Lead ID = Smith08cd492b-b11b-466a-85b9-324d58875f15
    

    Show image (as overlay)

    Also, on the App Connect dashboard if you look at the tile for flow you can see when the flow last ran successfully.

Conclusion

You have learned the process to build an ACE flow that can perform on-premises activities and call a flow in App Connect on IBM Cloud to perform on-cloud activities for SaaS applications, sharing data and processing between on-premises and on Cloud applications. You can take this same process to build more complex business solutions using the wide array of applications available in IBM App Connect.


Assets used in this demo

Project interchange file for App Connect Enterprise Toolkit

5 comments on"Sharing data and processing from on-premises activities with on-cloud SaaS applications using a callable on-cloud flow"

  1. Hi Team,

    I have downloaded the agentx.json file as mentioned in the steps. Copied the file in the config\iibswitch\agentx folder of my work directory. When I am trying to start the Integration server, I am getting the below error.

    …..2019-09-11 06:58:14.874924: .2019-09-11 12:28:14.897497: Integration server ‘ACEINTNODE’ starting initialization; version ‘11.0.0.5’ (64-bit)
    …………………………………..component starting: “agentx”
    ..Starting agentx with config folder: ‘C:\AppConnect_V11\workspace\Appconect_Callable_Workdirectotry\config\iibswitch\agentx’
    2019-09-11 12:28:21.129248: IBM App Connect Enterprise administration security is inactive.
    2019-09-11 12:28:21.182828: The HTTP Listener has started listening on port ‘7600’ for ‘RestAdmin http’ connections.

    2019-09-11 12:28:21.208368: Integration server has finished initialization.
    2019-09-11 12:28:23.802008: The connection agent for remote callable flows has encountered a network error when communicating with the Switch server with URL ‘wss://ibm-sw-ontqpeksebglffniz.eu-gb.ace.ibm.com:443/’. The error message is: ‘Error: unable to verify the first certificate’.
    AgentX: scheduling restart 0: 5000
    AgentX: restarting client connection 0
    AgentX: scheduling restart 0: 5000
    AgentX: restarting client connection 0

    • Hi Isha,
      There have been times when network issues interfere, and for which the easiest thing was to stop the agent and download a new config.

      Please stop your agent and then try to create a new agent (use “Connect callable flows” button to download a new agentx.json and then test the your agent). I just did this, and it worked OK for me.

      If creating a new agent works for you, please let me know.

      Otherwise, open a support ticket with your problem details, by using the links and info under “Access IBM Support” on the Support page.

      That will help the squad to investigate the problem properly, and if something is found to arrange a fix and/or doc update.

      Regards,
      Ian

      • Thanks for the suggestion. It worked.
        Also initially I was using a WIFI network. When I connected to a different network it worked.

  2. Hi IBM Team,

    I follow this lab but always getting the same error with same as Wrong Parameters:

    {“message”:”First Name: data value too large: Where – equals valueFirst Name – JohnLast Name – DoeEmail – j.doe@email.comCompany – Adeer.com (max length=40)”,”stack”:”STRING_TOO_LONG: First Name: data value too large: Where – equals valueFirst Name – JohnLast Name – DoeEmail – j.doe@email.comCompany – Adeer.com (max length=40)\n at HttpApi.getError (/home/vcap/app/node_modules/jsforce/lib/http-api.js:250:13)\n at /home/vcap/app/node_modules/jsforce/lib/http-api.js:95:22\n at tryCallOne (/home/vcap/app/node_modules/promise/lib/core.js:37:12)\n at /home/vcap/app/node_modules/promise/lib/core.js:123:15\n at flush (/home/vcap/app/node_modules/asap/raw.js:50:29)\n at /home/vcap/app/node_modules/async-listener/glue.js:188:31\n at _combinedTickCallback (internal/process/next_tick.js:132:7)\n at process._tickDomainCallback (internal/process/next_tick.js:219:9)\n at process.fallback (/home/vcap/app/node_modules/async-listener/index.js:565:15)”,”name”:”STRING_TOO_LONG”,”errorCode”:”STRING_TOO_LONG”,”fields”:[“FirstName”]}

    But, i just transfer 4 parameters as require:

    {
    “FirstName” : “John”,
    “LastName” : “Doe”,
    “Email” : “j.doe@email.com”,
    “Company”: “Adeer.com”
    }

    • Ian_Larner July 22, 2019

      @tèo,
      From the error message, I think you misinterpreted the instructions and typed or pasted all the Where condition details into one condition. I have worked through the instructions again, and made some updates because App Connect does more automatically for you now (since the instructions were last published).
      If you look at the sample configuration file provided for the flow, you can check what the fields should be configured as. If you continue to get problems, add another comment and we might work by email to look at your flow configuration.

      P.S. Sorry for the delay in replying; I’ve just got back from 2 weeks away.

      Regards,
      Ian

Join The Discussion

Your email address will not be published. Required fields are marked *