In IBM App Connect on IBM Cloud, Kibana logs are provided in the IBM Cloud Log Analysis service to help you monitor the behavior of your flows and diagnose errors. You can enhance logging by defining your own custom messages that you want App Connect to send to the logging service. You can also define custom fields for specific content that you want to log. This capability offers some control over what is written to the logs, so you can display information that’s important to you and locate your log entries more easily.

You can configure logging while creating or editing a flow by adding one or more Log nodes with a variety of log levels, custom messages, and custom fields. You can can add a Log node at any stage of the flow, after the event node that acts as a trigger. The Log node is available from the Toolbox tab in the flow editor.

Tip: You can also use a Log node to test your flow. For example, you can temporarily add a Log node to your flow to record some output from a previous node so that you can verify whether that node is correctly configured to provide the expected output.

Configuring logging

To add and configure a Log node within a flow, complete the following steps:

  1. At the stage in the flow where you want to capture relevant data from previous nodes or provide contextual information that should be written to the logs, click (+) and then from the Toolbox tab, click Log.

    Selecting the Log node from the Toolbox

    In the preceding example image, a ServiceNow “New incident” event is defined as the trigger for the flow, followed by a Set variable node with user-defined transactionId and priority data properties and mapped values. (The transactionId property is configured to resolve to a unique auto-generated value, and the priority value is calculated from the Impact and Urgency values recorded for the incident.)

    • View the “Set variable” mappings for the example

      Example of the Set variable node mappings

    In our example, we want to configure the Log node so that whenever a new incident is created, a message is written to the logs that a new incident is being processed. We’d also like to capture the transaction ID and priority for the incident.

  2. In the Log fields, configure the message that you want to pass to your App Connect logs:
    • Log level: Specify one of the following values:

      Value Description
      Info Log an informational message at that step in the flow.
      Error Log information about an unexpected situation or error in the flow.

      Tip: When such messages are emitted, they are recorded in the Kibana logs, but can also be viewed from the flow’s tile on the Dashboard by clicking the warning icon Flow error icon.

      • Example

        Example of an error icon on a flow tile and the corresponding message

      Debug Log detailed information for debugging purposes.

      Note: Debug messages are written to the logs only if debug logging is enabled on the flow. For information about how to enable debug logging, see the App Connect log levels section in the “Debugging flows in IBM App Connect” blog.

      Custom Specify a custom value (such as a JSONata mapping) that should resolve to Info, Error, or Debug. If left as Custom, the log level defaults to Info.

    • Message detail: Specify the message text that should appear in the log entry.

    In our example, we’ve defined a log level of Info, together with a custom message to indicate the new incident is being processed.

    Initial fields in the Log node

  3. If you want to define your own fields to add as Kibana index fields that are included in the log entry, add these custom fields as data properties with defined values.
    1. To define a data property, click Add property under Custom message fields to display the property editor. Then enter a property name and select its data type.
    2. Use Add property to define additional data properties.

      In the following example, we’ve defined two data properties that will contain a priority and a transaction ID for the incident.

      Example of defined properties under 'Custom message fields'

      Note: Only the top-level properties of the Custom message fields object will be added to the Kibana logs as index fields. A top-level property with nested properties will, however, display any data defined for its nested properties in the logs.

    3. To expose the properties as fields, click Edit mappings. You can now define values for these fields.

      Custom properties exposed as fields

    4. For each field, specify a value by mapping to fields from previous nodes in the flow or by entering text.

      In our example, we’ve mapped to the priority and transactionId data properties that we set up in the Set variable node because we want their values to be written to the logs.

      Custom properties exposed as fields with mapped values

      Tip:

      • You can add or remove properties by clicking Edit properties to switch back to the property editor.
      • If you want to update the name or data type of a defined property, ensure that the associated field is blank before you click Edit properties. (In the property editor, properties with defined values are grayed out and cannot be changed.)

You can complete your flow in the usual way by adding actions, additional Log nodes, or other toolbox utilities that you need. When the flow is triggered, you can view the logs to check the information recorded.

Viewing your custom messages and fields in the logs

To view the messages that are logged for your flow, open the App Connect menu Icon for hamburger menu, and select Manage > Logs. The Kibana logs for your App Connect instance should appear in the Discover page. (If necessary, change the default time filter from Last 15 minutes to a wider time range to extend your search results; for example, Last 1 hour.)

In the logs, you should see log entries for your flow, including an entry for the custom message that you defined in the Log node. This custom text (for example, Processing new ServiceNow incident) is displayed in the message column in the table. (If you don’t immediately see any log entries for your flow, wait a little while and check again.)

Initial view of Kibana logs

If you expand the log entry for the custom message, you’ll see the custom fields that you defined and their generated values. Notice that these fields are prefixed with the text user_ to differentiate them from the standard fields, and appended with _str by default (for example, user_incident_priority_str and user_transaction_id_str).

Expanded log entry for a custom message and custom fields

Now, reload the index fields list to include your custom fields so that you can search on them or add them as columns that you can sort by. Click Management > Index Patterns and then click the Refresh field list icon.

Refreshing the Kibana index

Click Discover to return to the logs. You should now see the custom fields under the Available Fields list. If you’d like to add these fields as columns to make the generated values more prominent, you can click the add button for each field.

Custom fields in the Kibana logs

Additional reference:
IBM Cloud Log Analysis documentation

Example

The following image builds on the example above and shows a sample flow that uses multiple Log nodes to apply different levels of logging in a flow and to generate custom messages. To read about what this flow does and see examples of the resulting messages in the Kibana logs, see the Example section in the “Enhanced user logging in IBM App Connect with the new Log node” blog.

Complete ServiceNow flow. (Click image to view full size.)

Join The Discussion

Your email address will not be published. Required fields are marked *