Overview

Skill Level: Any Skill Level

This recipe explains how to implement the event integration between CA UIM and Netcool using Json payload.

Ingredients

For this recipe the following prerequisites are expected:

  • an existing Netcool environment;
  • an existing CA UIM instance;
  • network flow between CA UIM and the Netcool systems for the Json payload (port number is configurable on the Netcool MessageBus probe side);
  • an available machine to install the Messagebus probe on the Netcool environment;
  • an existing CA UIM server to deploy the messagegtw gateway to.

Step-by-step

  1. Install the MessageBus Probe on the Netcool (in case it does not exists)

    The messagebus probe is used to receive JSON event messages from the CA APM solution.

    The probe needs to be installed and connected to the existing Netcool environment so received events can be used later on the current integrations in use, like ticketing, paging, emailing, automations and others.

    For more information on how to install the Netcool MessageBus Probe component, please refer to the documentation available on the following link.

    https://www.ibm.com/support/knowledgecenter/SSSHTQ/messbuspr-pdf.pdf

  2. Deploy the Messagegtw Gateway on the UIM

    The Messagegtw Gateway (also called as probe) is used to forward events from UIM to Netcool using Json format.

    This Gateway is currently not available on the UIM archive by default so it needs to be downloaded from CA portal and uploaded/imported to UIM archive before it can be deployed to a node.

    In case the UIM HUB has access to the internet follow steps below to get the messagegtw probe:
    1) login on to your UIM Web Admin Console;
    2) click on Archive;
    3) locate the messagegtw using the filter field;
    4) click on the 3 dots in front of the Gateway name and click on Download;
    5) once downloaded do same as shown above for deploying probe to a node (e.g. primaryhub), now using the Deploy option.

    In case the UIM HUB does not have access to the internet, follow steps below to get the probe:
    1) log in to https://support.nimsoft.com (it may requires an specific ID for login – ask CA team for access);
    2) inside nimsoft portal, click on archive and locate the messagegtw Gateway ;
    3) download the Gateway package to your desktop clicking on the save icon;
    4) login on to your UIM Web Admin Console;
    5) click on Archive and on Actions button;
    6) click on Import Packages and select the package downloaded on step 3;
    7) once loaded, locate the messagegtw using the filter field;
    8) click on the 3 dots in front of the Gateway name and click on Deploy;
    9) select the target node where the Gateway will be deployed (e.g. primaryhub)

  3. Configure the MessageBus Probe (Netcool side)

    The Probe has 2 important files, the props and the rules files.

    For the props, the most important informations are shown below. Other config lines are also required, but they are generic to all probe (like AuthUserName and AuthPassword), so not described in here.

    # Probe Specific Properties
    MessagePayload : ‘json’
    TransportFile : ‘$OMNIHOME/java/conf/httpWebhookTransport.properties’
    TransportType : ‘HTTP’
    RulesFile : ‘/opt/IBM/GSMA/probe/MsgBus/rules/gsma_eif.rules’

    For the rules, the GSMA EIF rules file was used for this integration. Please refer to the GSMA documentation for more information.

  4. Configure the Messagegtw Gateway (UIM side)

    To configure the Messagegtw Gateway following steps below:

    1) open Infrastructure Manager client (IM) and navigate to the messagegtw probe
    2) double click messagegtw to configure it
    3) on the configuration screen, navigate to listeners>webhook>publishers>1
    4) double click on the url key and update it accordingly to your Netcool message bus probe URL (e.g. http://<IP>:8080)
    5) update authentication/username/password accordingly based on your Netcool message bus probe authentication requirements
    6) on the configuration screen, navigate to listeners>webhook>publishers>config>payload
    7) double click on the “content” key and update with below payload as the value for that key:
    {“origin” : “${cs.origin}”, “ComponentType”: “${message.typeName}”, “Severity” : “${message.udata.level}”, “NodeAlias” : “${message.DisplayName}”, “InstanceId” : “${message.InstanceId}” , “hostname” : “${cs.name}” , “Component” : “${message.udata.prid}”, “InstanceSituation” : “${message.InstanceSituation}” , “ClassName” : “${message.ClassName}”, “source” : “CA_UIM” , “LastOccurence” : “${message.LastOccurence}” , “InstanceValue” : “${ci.name}” , “appl_id” : “${message.udata.prid}”, “SubComponent” : “${message.udata.subsys}” , “msg” : “${message.udata.message}” , “AlertKey” : “${message.udata.alarmid}”}
    7b) optionally instead of creating 1 key (content) with multiple sub-keys as value, you can create individual keys with their values by clicking on the New Key button and adding all the required pairs (keys/values) than delete the “content” key. Like pairs (Key = Value) below:
    origin = “${cs.origin}”
    ComponentType = “${message.typeName}”
    Severity = “${message.udata.level}”
    NodeAlias = “${message.DisplayName}”
    InstanceId = “${message.InstanceId}”
    hostname = “${cs.name}”
    Component = “${message.udata.prid}”
    InstanceSituation = “${message.InstanceSituation}”
    ClassName = “${message.ClassName}”
    source = “CA_UIM”
    LastOccurence = “${message.LastOccurence}”
    InstanceValue = “${ci.name}”
    appl_id = “${message.udata.prid}”
    SubComponent = “${message.udata.subsys}”
    msg = “${message.udata.message}”
    AlertKey = “${message.udata.alarmid}”
    8) on the configuration screen, navigate to listeners>webhook
    9) create new property key named preprocessing_script with value netcool.groovy
    10) copy netcool.groovy file (find file content in the section 7) to your primaryHub linux box under /opt/nimsoft/probes/gateway/messagegtw/ directory. This netcool.groovy file is responsible for pre-processing the events and generate new values to be used on the payload.
    11) click apply then close probe configuration window
    12) restart the messagegtw to reread this new configuration.

     

    Please find more details at https://docops.ca.com/ca-unified-infrastructure-management-probes/ga/en/alphabetical-probe-articles/messagegtw-message-gateway/messagegtw-message-gateway-ac-configuration

  5. Configure the HUB Probe (UIM side)

    To configure the HUB Probe queue in order to allow messages to correctly flow to Messagegtw Gateway, follow steps below:

    1) open IM and navigate to the hub probe
    2) double click hub probe to configure it and click on the Queues tab
    3) select webhook queue and click delete button to delete existing queue
    4) right click and click on new to create a new queue
    5) provide the queue name and subject as webhook and select type as attach
    6) click apply and ok

  6. Configure NAS Probe (UIM side)

    To configure the NAS Probe Auto-Operator in order to trigger messages to flow to Messagegtw Gateway, follow steps below:

    1) open IM and navigate to the nas probe
    2) double click nas probe to configure it and click on the Auto-Operator/Profiles tab
    3) inside profiles tab, right click and select new to create a new
    4) select action type as webhook from list
    5) select existing path to message probe
    6) select existing webhook name
    7) select required severity and put * in message string field
    8) select action mode as on message arrival
    9) click ok
    10) put name as Netcool and click ok
    11) click apply and ok

  7. netcool.groovy (copy content below into /opt/nimsoft/probes/gateway/messagegtw/netcool.groovy)

    import com.nimsoft.nimbus.PDS
    import com.nimsoft.nimbus.NimUtility
    import com.nimsoft.cet.utility.SimpleLogger
    import com.nimsoft.nimbus.NimRequest

    // extract some regularly needed data for easier access
    String subject = message.getString(“subject”, “unknown”) // default to “unknown” in case there is no subject
    String probe = message.getPDS(“udata”).getString(“prid”, “noprobe”)
    String origin = message.getPDS(“udata”).getString(“origin”, “unknown”)
    log.debug(“netcool.groovy: origin” + origin )

    // look up for typename
    if( message.getPDS(“udata”).getString(“dev_id”, null) != null) {
    String devid = message.getPDS(“udata”).getString(“dev_id”)
    String typeName = cache.get(“select csa.cs_attr_value from cm_device d join cm_computer_system_attr csa on d.cs_id=csa.cs_id where d.dev_id='” + devid + “‘ and csa.cs_attr_key=’typeName'”)
    log.debug(“netcool.groovy: typeName ” + typeName)
    message.putString(“typeName”, typeName)
    }

    // look up for DisplayName
    if( message.getPDS(“udata”).getString(“dev_id”, null) != null) {
    String devid = message.getPDS(“udata”).getString(“dev_id”)
    String DisplayName = cache.get(“select csa.cs_attr_value from cm_device d join cm_computer_system_attr csa on d.cs_id=csa.cs_id where d.dev_id='” + devid + “‘ and csa.cs_attr_key=’DisplayName'”)
    log.debug(“netcool.groovy: DisplayName ” + DisplayName)
    message.putString(“DisplayName”, DisplayName)
    }
    // look up for ClassName
    if( message.getPDS(“udata”).getString(“dev_id”, null) != null) {
    String devid = message.getPDS(“udata”).getString(“dev_id”)
    String ClassName= cache.get(“select p.probe_group from cm_nimbus_robot r left join cm_nimbus_probe p on r.robot_id=p.robot_id where r.dev_id='” + devid + “‘”)
    log.debug(“netcool.groovy: ClassName ” + ClassName)
    message.putString(“ClassName”, ClassName)
    }
    // look up for InstanceId
    if( message.getPDS(“udata”).getString(“dev_id”, null) != null) {
    String met_id = message.getPDS(“udata”).getString(“met_id”)
    String InstanceId = cache.get(“select csa.ci_id from cm_configuration_item_metric csa where csa.ci_metric_id='” + met_id + “‘”)
    log.debug(“netcool.groovy: InstanceId” + InstanceId)
    message.putString(“InstanceId”, InstanceId)
    }
    // look up for InstanceValue
    if( message.getPDS(“udata”).getString(“dev_id”, null) != null) {
    String devid = message.getPDS(“udata”).getString(“dev_id”)
    String InstanceValue = cache.get(“select ci.ci_name from cm_configuration_item ci where ci.dev_id='” + devid + “‘”)
    log.debug(“netcool.groovy: InstanceValue ” + InstanceValue)
    message.putString(“InstanceValue”, InstanceValue)
    }
    // look up for InstanceSituation
    if( message.getPDS(“udata”).getString(“message”, null) != null) {
    String messagealarm = message.getPDS(“udata”).getString(“message”)
    String InstanceSituation = messagealarm.substring(0,10);
    log.debug(“netcool.groovy: InstanceSituation ” + InstanceSituation)
    message.putString(“InstanceSituation”, InstanceSituation)
    }
    // look up for LastOccurence
    if( message.getPDS(“udata”).getString(“nimts”, null) != null) {
    String occurencemills = message.getPDS(“udata”).getString(“nimts”)
    long timeinsec = Long.parseLong(occurencemills);
    Date date2 = new Date(timeinsec * 1000 );
    log.debug(“netcool.groovy: LastOccurence” + date2.toString())
    message.putString(“LastOccurence”, date2.toString())
    }

    if( subject.matches(“.*alarm.*”) ) {
    String level = message.getPDS(“udata”).getString(“level”, “0”)
    int eisLevel = 0
    String eisStatus = “Active”
    if(“0”.equals(level)) {
    eisLevel = 5
    eisStatus = “Closed”
    } else if(“1”.equals(level)) {
    eisLevel = 5 // Normal
    eisStatus = “Active”
    } else if(“2”.equals(level)) {
    eisLevel = 3 // minor
    eisStatus = “Active”
    } else if(“3”.equals(level)) {
    eisLevel = 3 // minor
    eisStatus = “Active”
    } else if(“4”.equals(level)) {
    eisLevel = 2 // major
    eisStatus = “Active”
    } else if(“5”.equals(level)) {
    eisLevel = 1 // critical
    eisStatus = “Active”
    } else {
    eisLevel = 3
    eisStatus = “Active”
    }
    message.putInt(“eis_severity”, eisLevel)
    message.putString(“eis_status”, eisStatus)
    }

    // we set a “eis token” which is the i18n_token without the prefixed “as#”
    String eistoken = message.getPDS(“udata”).getString(“i18n_token”, “as#unknown”).substring(3)
    message.putString(“eis_token”, eistoken)

    String msg = NimUtility.displayPDS(message)

    // real message processing starts here

     

    String usertag1 = message.getString(“user_tag_1”, null)
    if(usertag1 != null && isValidEsid(usertag1) ) {
    // ok, there is a ESID in the user tag 1 already, copy it into the esid field and we’re done!
    message.putString(“esid”, usertag1 )
    log.debug(“alarm.groovy: ESID found through user tag (Rule 1)”)
    return message
    } else {
    // no ESID in the event. Now we apply all of our various lookup rules.
    log.debug(“alarm.groovy: no ESID found in the event, starting lookups: ” + msg)

    if( probe.matches(“(snmptd|snmpcollector|pollagent|net_connect|interface_traffic|clariion|celerra|netapp_ontap)”) ) {
    /*
    * for networking and storage probes, we will look up by ipaddress. In alarms, the ip address is stored in udata.values.ip
    * for QOS_MESSAGEs however, the ip address is not in the message at all.
    * it seems easiest to just use the ip address associated with the CM_DEVICE perspective in both cases and to join that with the replica table.
    */
    log.debug(“alarm.groovy: Attempting management ip address lookup (Rule 2)”)
    String devid = message.getPDS(“udata”).getString(“dev_id”, “”)
    String esid = cache.get(“select r.esid from cm_device d join dd_itsm_replica r on r.conn_object_type in (‘snmpv2′,’snmpv3′,’clariion’,’ssh’,’ping’,’tcp’,’celerra’,’netapp’) and r.lookup_value=d.dev_ip where r.origin='” + origin + “‘ and d.dev_id='” + devid + “‘”)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through management ip address lookup (Rule 2)”)
    return message;
    }

    } else if( probe.equals(“cisco_ucs”) ) {
    /*
    * For cisco_ucs, we use the data from the suppression key. Examples for suppression keys are:
    * “ESC:ETHERPIO:10.250.10.11::sys/switch-B/slot-1/switch-ether/port-14″.”operState”
    * “ESC:CHA_COMPUTEBLADE:10.250.10.11::sys/chassis-1/blade-2″.”fmTempSenIo”
    *
    * initially, we will try to look up by ipaddress and switch/server id.
    */
    String suppressionKey = message.getPDS(“udata”).getString(“supp_key”, null)
    if(suppressionKey != null) {
    String elementMgrIp = suppressionKey.replaceAll(“.*?:(\\d+\\.\\d+\\.\\d+\\.\\d+)::(sys/switch-\\S/|sys/chassis-\\d/blade-\\d).*”, “\$1”);
    String componentName = suppressionKey.replaceAll(“.*?:(\\d+\\.\\d+\\.\\d+\\.\\d+)::(sys/switch-\\S/|sys/chassis-\\d/blade-\\d).*”, “\$2”);
    String esid = cache.get(“select r.esid from dd_itsm_replica r where r.conn_object_type=’cisco_ucs’ and r.lookup_value='” + elementMgrIp + ‘::’ + componentName + “‘ and r.origin='” + origin + “‘”)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through cisco_ucs element mgr ip and component name (Rule 3.1)”)
    return message;
    } else {
    /*
    * If we cannot find anything, the component is not managed. Let’s see if the element manager itself is managed.
    */
    log.debug(“alarm.groovy: cisco ucs component is not managed. Attempting lookup for element manager (Rule 3.2)”)
    esid = cache.get(“select r.esid from dd_itsm_replica r where r.conn_object_type=’cisco_ucs’ and r.lookup_value='” + elementMgrIp + “‘ and r.origin='” + origin + “‘”)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through cisco_ucs element mgr ip fallback (Rule 3.2)”)
    return message;
    }
    }
    }
    } else if( probe.equals(“vmware”)) {
    String devId = message.getPDS(“udata”).getString(“dev_id”, “”)
    String sql = “select case when r1.esid is null then r2.esid else r1.esid end as esid \
    from cm_device d join cm_device_attribute da1 on da1.dev_id=d.dev_id and da1.dev_attr_key=’PrimaryRole’ \
    join cm_device_attribute da2 on da2.dev_id=d.dev_id and da2.dev_attr_key=’VirtualManagerId’ \
    join cm_device_attribute vcda1 on da2.dev_attr_value=vcda1.dev_attr_value \
    join cm_device_attribute vcda2 on vcda2.dev_attr_key=’PrimaryRole’ and vcda2.dev_attr_value=’vCenter’ and vcda2.dev_id=vcda1.dev_id \
    join cm_device vcd on vcd.dev_id=vcda2.dev_id \
    left join dd_itsm_replica r1 on \
    (lower(r1.conn_object_type)=lower(da1.dev_attr_value) and r1.lookup_value=concat(vcd.dev_name,’::’, d.dev_name)) \
    left join dd_itsm_replica r2 on \
    ( r2.conn_object_type=’vcenter’ and r2.lookup_value=vcd.dev_name) \
    where d.dev_id='” + devId + “‘ limit 1”
    log.debug(“alarm.groovy: vmware lookup query: ” + sql )
    // need to find out what to lookup by
    String esid = cache.get(sql)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through vmware lookup (Rule 4)”)
    return message;
    }
    } else if( probe == “azure” ) {
    // need to find out what to lookup by
    String esid = cache.get(“”)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through management ip address lookup (Rule 5)”)
    return message;
    }
    } else if( probe == “hyperv” ) {
    // need to find out what to lookup by
    String esid = cache.get(“”)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through management ip address lookup (Rule 6)”)
    return message;
    }
    } else if( probe == “xenserver” ) {
    // need to find out what to lookup by
    String esid = cache.get(“”)
    if(!””.equals(esid) && isValidEsid(esid)) {
    message.putString(“esid”, esid)
    log.debug(“alarm.groovy: ESID found through management ip address lookup (Rule 8)”)
    return message;
    }
    }

    }

    return message;

     

    boolean isValidEsid(String input) {
    //return input.matches(“^[A-Za-z][A-Za-z][A-Za-z]-[A-Za-z][A-Za-z][0-9]+\$”)
    return input.matches(“^\\S+\$”)
    }

Join The Discussion