IBM App Connect Professional provides several techniques for you to move bulk data (thousands of records) from SAP to Salesforce.com. Using these techniques, you can create or update hundreds of Salesforce objects in one API request, making the most of the Salesforce.com limits for the total API requests per 24-hour period.

For details about Salesforce API request limits, see API Usage Metering in the Salesforce Developer Documentation.

For example, you might have many customer records in SAP on your own private network. Your company might be moving to use Salesforce.com as your customer relationship management (CRM) system in the cloud, and want to migrate the customer data from SAP to Salesforce.com. Alternatively, your company might have decided to continue using SAP as the master for customer records but on occasion need to a bulk data sync of changes to many records in SAP with corresponding objects in Salesforce.com.

To move bulk data from SAP to Salesforce with IBM App Connect Professional Studio, you can use the following techniques:

  • Split the load over separate days. For example, use a filter activity to split the customers into separate groups that can be validated and upserted into Salesforce on separate days.
  • Do multiple updates in one activity. The upsert activity allows you to update multiple Salesforce objects in one API request, as described in Upserting multiple Salesforce objects in one API request.

    While the Salesforce.com API limits the number of objects that it can create in a single upsert call to 200 objects, the App Connect Salesforce.com connector does not have such a limit. You can use one Upsert Objects activity to create an unlimited number of objects.

  • Use the Salesforce.com bulk API. Salesforce.com provides a bulk API for asynchronous updates. The bulk API allows you to send a bulk request in one call and then retrieve the response within another call, as described in Using Salesforce.com bulk API to create multiple Salesforce objects in one API request.

You can also move bulk data from SAP to Salesforce without using the SAP connectivity in App Connect Professional. For example, you can export data from SAP to a CSV file, and then use App Connect Professional to migrate the data from the CSV file to Salesforce.com’s cloud offerings. For example, using the following tutorial with a CSV file of customer data exported from a SAP: Migrating account data from a flat file to Salesforce.com using IBM App Connect Professional

Upserting multiple Salesforce objects in one API request

Consider the following simplified flow:

Using Upsert Objects activity to update multiple Salesforce objects in one API call. (Click image to view full size.)

The flow contains the following activities:

  • The Receive Request activity.
    This activity receives a request with or without a search criteria and triggers the orchestration.
  • The SAP activity Get Customer List.
    This activity returns a recurring element with all customers that match the criteria. With the customer number, it also returns the basic data, such as name, address, and so on.
  • The Split activity.
    Although SAP can return hundreds of account records in one chunk, it does not make sense to send them in one chunk synchronously to Salesforce.com. So, you can use the Split activity to split the recurring element into chunks of an appropriate size (for example 20, 50, or 100 records). The Split activity is similar to a loop but returns a recurring element with a subset of the original elements.

    Using a Split activity to split a recurring element into chunks. (Click image to view full size.)
  • The Filter and Profile activity.
    The Filter activity validates multiple datasets in one step. The Filter activity returns a recurring element for valid records, a recurring element for invalid records, and statistics.

    Using a Filter activity to validate multiple datasets in one step. (Click image to view full size.)

    If you want to do more complex validation, you can replace the Filter activity with a loop that validates each record separately and that flags each record as good or bad. Then, filter the good records and use only those records for the Upsert activity.

  • The Upsert activity.
    The Upsert activity inserts or updates multiple accounts in Salesforce.com in one call. The activity returns a recurring element that contains details for each account record whether or not the upsert was successful.

Using Salesforce.com bulk API to create multiple Salesforce objects in one API request

Consider the following extract of the flow from the template T0314_SAPToSalesForce_BidiSync:

Using the Salesforce.com bulk API to create multiple Salesforce objects in batches. (Click image to view full size.)

The flow contains the following activities:

  • The Receive Request activity.
    This activity receives a request and triggers the orchestration.
  • The SAP activity Get Customer List.
    This activity returns a recurring element with all customers that match the criteria, up to the maximum number of contacts defined by the SapSearchCriteriaMaxRows configuration property of the integration project. With the customer number, it also returns the basic data, such as name, address, and so on.
  • Batch Insert to SFDC Contacts.
    This activity creates a batch insert job for the Salesforce.com bulk API to insert multiple Contact records into Salesforce in batches. The activity splits the input recurring element into batches of up to 10,000 records.
  • Get Batch Status.
    This activity gets information about an existing batch job, returning one or more batchinfo objects depending on the number of records processed by the associated job. Each batchinfo object contains the state of the batch and informs you whether you should proceed to get the results or whether you need to wait or fix errors related to your request.
  • Get Batch Results.
    This activity gets the results of a batch job that has finished processing. The flow can process those results further, such as to write data to a log message.

For more information about using the Salesforce.com Bulk API, see Loading Large Data Sets with the Force.com Bulk API in the Salesforce Developer Documentation.

Join The Discussion

Your email address will not be published. Required fields are marked *