Ever have to follow a complicated chain of CICS tasks through multiple regions to find out why an application is hanging? Or had to work out the origin of a problem task? If so, transaction tracking is what you need!
CICS transaction tracking shows the origin of and the relationships between tasks in an application as they flow across CICS regions in a CICSplex. Take an example of users suddenly reporting their application is hanging. You know processing is via a web service request into CICS, and that this web service makes use of default pipeline processing, so you use the Tasks view in CICS Explorer to find all the instances of the CPIH pipeline transaction:
Look at those suspend times, what is going on? Double clicking on the oldest instance of CPIH (task 494 on region IYK3ZAB4) to obtain its attributes and filtering on suspend shows:
so you can see that the work is suspended waiting on a response over IPCONN AB03 which is connected to region IYK3ZAB3. OK, now you know to go investigating tasks running on IYK3ZAB3. But wait! No need with transaction tracking, there’s a simpler way. Right clicking this top instance of CPIH gives you the option to Search Associated tasks:
and this shows what tasks are associated with this instance of CPIH in the Search results tab:
So now you know that the associated mirror task on IYK3ZAB3 is task 3142 which itself routed on to mirror task 2574 on region IYK3ZAB5. By default the Search results tab shows the suspend reason for each task, so you can immediately see that it is suspended on TSMAINLM and so know to check what is using all the temporary storage on IYK3ZAB5 and address this.
Transaction tracking allowed you to find the reason for the hang in just a couple of clicks!
A closer look:
How does this work? Transaction tracking is based on the concept of every user task in a CICSplex having a point of origin e.g. a web service request or an MQ message. When work first enters the CICSplex, details of its point of origin is placed into task context information called “origin data”, part of its task association data, for the first task that is created to process it. This data, the origin data, flows with the work as it moves around the CICSplex.
Looking at the above example again, the origin data for the hanging mirror task can be obtained by right clicking on it in the Search results tab and selecting “Task Association” then “Open”:
This shows that the origin of the work in CICS was indeed the task mentioned above, pipeline handler task 494 on region IYK3ZAB4 for URIMAP $649090, and the web service request came in to CICS from IP address 126.96.36.199 port 57145 for user ID ALISONB. If it was the hanging mirror that had been noticed then this information could be used to track back to the owner of the work – searching on this mirror’s associated tasks would give the same results as searching from the pipeline handler task.
What about work not initiated by web service requests? What information is available to identify their points of origin? This depends on the origin itself. Work from MQ for example makes use of adapter data fields of the origin data. Their contents depends on how the work was initiated; for work initiated by the MQ Trigger monitor as a result of a message being put onto application queue ALITRIG1 on qmgr MQD6 you’d see:
In CICS 5.3 we enhanced the MQ Bridge to provide similar information, but of course without any initiation queue name and the bridge queue name in adapter data 3.
Work from the CICS Sockets Listener provides the IP addresses and port numbers of the local and remote session partners (from z/OS V2R2 Communications Server) and work from CICS transaction gateway can even be configured to pass on the root and current Cross Component Trace (XCT) contexts. You can even tag the work yourself by making use of the XAPADMGR global user exit.
In summary, transaction tracking shows the origin of and the relationships between tasks in an application as they flow across CICS regions in a CICSplex. Transaction tracking information is written out to SMF in a task’s monitoring data and can be useful not just for problem determination but for audit purposes too. For more details on transaction tracking and the information that is available see here.