Introduction

As you prepare to deploy APIs to z/OS Connect EE, you might be considering how to measure the number of API requests, and monitor response times and CPU cost. In this article we’ll give an example of how to use z/OS Workload Management (WLM) to classify API requests and Resource Management Facility (RMF) to measure z/OS Connect EE API workloads.

Using WLM to classify API requests

WLM uses classification rules to associate incoming requests with service classes which specify appropriate performance goals. z/OS Connect workloads are managed by two types of WLM classification:

  • STC type classification
  • z/OS Connect EE servers can be classified as STCs (started tasks) in order to manage all of the server activity that does not relate to specific transactions, for example, garbage collector activity and connection pool management.
    Note: A z/OS Connect EE server can also be started as a submitted job, in which case you would use an equivalent JES classification.

  • CB type classification
  • Every request to a z/OS Connect EE server is dispatched on a USS (Unix System Services) thread and executes under a WLM enclave that is classified as a CB workload transaction.

The processor activity of z/OS Connect EE transactions is reported to WLM and RMF and so can be measured in RMF Workload Activity reports for service classes and report classes. With the recent WLM enhancements you can now also tag z/OS Connect EE requests with MOBILE reporting attribute (see article ‘Limiting the impact of API workloads on MLC’).

A transaction class provides a granular approach for classifying API requests. For instance, you can classify requests for different API operations inside the same z/OS Connect EE server depending on the request URI.

A transaction class for z/OS Connect EE is enabled using the zosWlm-1.0 feature. After adding this feature, validate that the z/OS Connect EE server is authorized to use the feature by checking for the message ‘CWWKB0103I: Authorized service group ZOSWLM is available’ in the messages log. For more information on authorizing z/OS Connect EE to use WLM, refer to
https://www.ibm.com/support/knowledgecenter/en/SS7K4U_liberty/com.ibm.websphere.wlp.zseries.doc/ae/twlp_wlmclassification.html

Using a transaction class to classify API requests

Transaction classes are defined in the server.xml file. Example 1 shows an example classification of 3 API operations for the sample catalog API. The catalog API is used to access the CICS Catalog Manager sample application. For more information on the catalog API, refer to the z/OS Connect EE Getting Started Guide.

Example 1 Transaction class definition in server.xml

  
<wlmClassification>                                                    
  <httpClassification transactionClass="TCIC" method="GET" resource="/catalogManager/items"/>
  <httpClassification transactionClass="TCIS" method="GET" resource="/catalogManager/items/*"/>
  <httpClassification transactionClass="TCPO" method="POST" resource="/catalogManager/orders"/>
</wlmClassification>  
 

In Example 1 we assign transaction class TCIC to the inquire catalog API operation which is invoked using an HTTP GET request with URI /catalogManager/items. We also assign transaction classes to the inquire single and place order API operations.

For more information on defining a transaction class refer to
https://www.ibm.com/support/knowledgecenter/SS7K4U_liberty/com.ibm.websphere.wlp.zseries.doc/ae/rwlp_wlmclassification.html

Example 2 shows the corresponding WLM classification rules for classifying the catalog API requests.

Example 2 API WLM classification rules

                  
Subsystem Type . : CB          Fold qualifier names?   Y  (Y or N)     
Description  . . . WebSphere requests                                  
                                                                       
Action codes:   A=After     C=Copy        M=Move     I=Insert rule     
                B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule
                                                             More ===> 
         -------Qualifier--------                 -------Class-------- 
Action   Type      Name     Start                  Service     Report  
                                         DEFAULTS: CBENCLAV    RCBDEF  
_____  1 CN        MOPZCET  ___                    CBHI        RCBZCET
_____  2   TC        TCIC     ___                  CBHI        RCBTCIC
_____  2   TC        TCIS     ___                  CBHI        RCBTCIS
_____  2   TC        TCPO     ___                  CBHI        RCBTCPO       

Example 2 defines specific report classes to monitor the different API operations, for example, transaction class TCIC for inquire catalog requests is assigned to the RCBTCIC report class. We will use this report class to report on the number of inquire catalog API operations, the response time and CPU consumed.

Classifying the z/OS Connect EE server

Example 3 shows the WLM classification rule for classifying the z/OS Connect EE server.

Example 3 z/OS Connect EE server WLM classification rule

                 
Subsystem Type . : STC         Fold qualifier names?   Y  (Y or N)     
Description  . . . Started Tasks classifications                       
                                                                       
Action codes:   A=After     C=Copy        M=Move     I=Insert rule     
                B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule
                                                             More ===> 
         -------Qualifier--------                 -------Class-------- 
Action   Type      Name     Start                  Service     Report                     
                                         DEFAULTS: OPSDEF      RSTCDEF 
____  1  TN        MOPZCET  ___                    SYSSTC      RSTCZCET   

Example 3 defines a specific report class RSTCZCET to monitor the z/OS Connect EE started task (STC) MOPZCET. We will use this report class to report on the server activity that does not get attributed to specific API requests.

Measuring the API

Example 4 shows an extract from the RMF Workload Activity report for inquire catalog API requests (report class RCBTCIC).

Example 4 RMF Workload Activity report for report class inquire catalog API requests

                 
REPORT BY: POLICY=WLMPOL  REPORT CLASS=RCBTCIC         
                          DESCRIPTION =CB RC for inquire catalog API 

-TRANSACTIONS-  TRANS-TIME HHH.MM.SS.TTT  SERVICE TIME  ---APPL %---  
AVG       0.04  ACTUAL                 2  CPU    7.176  CP      1.20
MPL       0.04  EXECUTION              1  SRB    0.000  AAPCP   0.00
ENDED    11900  QUEUED                 0  RCT    0.000  IIPCP   1.20
END/S    19.83  R/S AFFIN              0  IIT    0.000               
£SWAPS       0  INELIGIBLE             0  HST    0.000  AAP      N/A
EXCTD        0  CONVERSION             0  AAP      N/A  IIP      N/A
AVG ENC   0.04  STD DEV                0  IIP      N/A     

TRANSACTION APPL% :   TOTAL :  CP   1.20   AAP/IIP ON CP   1.20   AAP/IIP
                     MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP 

Example 4 provides us with the following useful reporting information:

  • An average of 19.83 inquire catalog API requests were executed per second during the recording interval (10 minutes)
  • A total of 7.176 seconds of CPU time was consumed by the inquire catalog API requests, which equates to 0.6ms per request
  • The total CPU consumed is equivalent to 1.20% of a CP
  • 100% of the CPU consumed is zIIP eligible
    Note:The IIPCP value represents the percentage of a CP that could have run on a zIIP if a zIIP had been available on the system.

  • No mobile CPU is reported since we have not classified the inquire catalog API requests as being initiated by a mobile device

Example 5 shows a similar extract from the RMF Workload Activity report for inquire single API requests (report class RCBTCIS).

Example 5 RMF Workload Activity report for report class inquire single API requests

                 
REPORT BY: POLICY=WLMPOL  REPORT CLASS=RCBTCIS         
                          DESCRIPTION =CB RC for inquire single API 

-TRANSACTIONS-  TRANS-TIME HHH.MM.SS.TTT  SERVICE TIME  ---APPL %---  
AVG       0.03  ACTUAL                 1  CPU    3.930  CP      0.65
MPL       0.03  EXECUTION              1  SRB    0.000  AAPCP   0.00
ENDED    11900  QUEUED                 0  RCT    0.000  IIPCP   0.65
END/S    19.83  R/S AFFIN              0  IIT    0.000               
£SWAPS       0  INELIGIBLE             0  HST    0.000  AAP      N/A
EXCTD        0  CONVERSION             0  AAP      N/A  IIP      N/A
AVG ENC   0.03  STD DEV                0  IIP      N/A     

TRANSACTION APPL% :   TOTAL :  CP   0.65   AAP/IIP ON CP   0.65   AAP/IIP N/A
                     MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP N/A 

Example 5 shows that a total of 3.930 seconds of CPU time was consumed by the inquire single API requests, which equates to 0.33ms per request. Note that this is less than the average CPU cost of inquire catalog API requests because an inquire single request retrieves a single item from the catalog, whereas the inquire catalog request retrieves 15 items.

Reporting on the z/OS Connect EE server

Example 6 shows an extract of an RMF Workload Activity report for report class RSTCZCET. This is the report class that we defined in Example 3 for the z/OS Connect EE server.

Example 6 RMF Workload Activity report for z/OS Connect server

                 
REPORT BY: POLICY=WLMPOL  REPORT CLASS=RSTCZCET        
                          DESCRIPTION =STC classification for MOPZCET

-TRANSACTIONS-  TRANS-TIME HHH.MM.SS.TTT  SERVICE TIME  ---APPL %--- 
AVG       1.00  ACTUAL                 0  CPU    5.158  CP      0.92
MPL       1.00  EXECUTION              0  SRB    0.339  AAPCP   0.00 
ENDED        0  QUEUED                 0  RCT    0.000  IIPCP   0.85
END/S     0.00  R/S AFFIN              0  IIT    0.000             
£SWAPS       0  INELIGIBLE             0  HST    0.000  AAP      N/A
EXCTD        0  CONVERSION             0  AAP      N/A  IIP      N/A
AVG ENC   0.00  STD DEV                0  IIP      N/A   

TRANSACTION APPL% : TOTAL :   CP   0.92   AAP/IIP ON CP   0.85   AAP/IIP  N/A
                    MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP  N/A

Example 6 shows that a total of 5.158 seconds of CPU time was consumed by the server during the reporting interval (10 minutes). The total CPU consumed by the server is equivalent to 0.92% of a CP, out of which 0.85% of the CP is zIIP eligible.

Measuring the CICS transaction

Each request to the catalog API results in the execution of a CICS transaction. We also use WLM to classify the CICS transaction and measure the transaction response time and CPU consumption.

Note: Starting with CICS TS V5.3, transactions can now be classified into service and report classes, which directly measure and report their CPU consumption. The same support is available with IMS V14.

Example 7 shows the WLM classification rules for classifying the CICS Catalog Manager transactions.

Example 7 CICS transaction classification rules

                 
Subsystem Type . : CICS        Fold qualifier names?   Y  (Y or N)     
Description  . . . CICS Environment                                  
                                                                       
Action codes:   A=After     C=Copy        M=Move     I=Insert rule     
                B=Before    D=Delete row  R=Repeat   IS=Insert Sub-rule
                                                             More ===> 
         -------Qualifier--------                 -------Class-------- 
Action   Type      Name     Start                  Service     Report  
                                         DEFAULTS: CBENCLAV    RCBDEF  
_____  1 SI        CICSMOBT   ___                  CICSHI      RCICMOBT
_____  2   TN        MZIC     ___                  CICSHI      RCICMZIC
_____  2   TN        MZIS     ___                  CICSHI      RCICMZIS
_____  2   TN        MZPO     ___                  CICSHI      RCICMZPO

Example 7 defines specific report classes to monitor the different CICS transactions, for example, inquire catalog requests invoke CICS transaction MZIC which is classified in the RCICMZIC report class. We use this report class to report on the number of MZIC transactions, the response time and CPU consumed.

Example 8 shows an extract of an RMF Workload Activity report for report class RCICMZIC.

Example 8 RMF Workload Activity report for CICS transaction MZIC

                 
REPORT BY: POLICY=WLMPOL         REPORT CLASS=RCICMZIC        
                                 DESCRIPTION =RC for CICS transaction MZIC
                                                                              
-TRANSACTIONS-  TRANS-TIME HHH.MM.SS.TTT                                      
AVG       0.00  ACTUAL                 0                                      
MPL       0.00  EXECUTION              0                                      
ENDED    11900  QUEUED                 0                                      
END/S    19.83  R/S AFFIN              0                                      
                                                                              
TRANSACTION APPL% : TOTAL :   CP   0.39   AAP/IIP ON CP   0.00   AAP/IIP N/A    
                    MOBILE :  CP   0.00   AAP/IIP ON CP   0.00   AAP/IIP N/A    

Example 8 shows that an average of 19.83 MZIC transactions were executed per second, with an average response time of 0ms (due to rounding). It also shows that 0.39% of a CP was consumed by these transactions.

Note: In the same way that we use an STC WLM classification in Example 6 for the z/OS Connect EE server, you can also use an STC classification to report on the CICS server activity.

Summary

This article gives an example of how WLM and RMF can be used to measure the performance of z/OS Connect EE APIs. Many factors can influence the performance of APIs, including message lengths, security and back end connectivity.

More information

More information on using WLM to measure z/OS workloads can be found in the ITSO Redbooks publication Using IBM z/OS WLM to Measure Mobile and Other Workloads (REDP-5359-00).

More information on z/OS Connect EE performance can be found in the IBM z/OS Connect Enterprise Edition Performance Reports.

6 comments on"Measuring API workloads with WLM"

  1. Hi Nigel, If we classify z/OS Connect EE API requests under CB subsystem in WLM, Does the response CPU time reported by RMF Workload Activity include the CICS and DB2 CPU service time consumed by the request? Or we have to classify CICS transaction and DB2 seperately in WLM to get the time that the request spends in these subsystems?

    • Nigel Williams September 13, 2017

      Hi Joel. The average response time (ACTUAL) reported in the workload activity report includes the CICS transaction or DB2 response time. It is basically the elapsed time between the request arriving at z/OS Connect EE and the response being sent.

  2. Hey Nigel, Great summary of the functionality. A few questions:
    1. Do we know the overhead of the WLM definitions, classifications, et al? Wondering if there is a need to avoid defining too many classes
    2. If I did not define the enclaves … where does RMF report the resource utilization? ie: would it show up under zCEE, CICS, or would it be somewhat lost in the workload activity reports? Thanks,

    • NigelWilliams December 03, 2017

      Hi Mike.
      For #1, i’m not aware of any overhead in defining WLM report classes for different Liberty transactions i.e API requests in the case of z/OS Connect. However, I will check with my colleagues who are the WLM experts.

      For #2, if you do not define transaction classes for Liberty requests (using the zosWlm-1.0 feature) all of the CPU consumption will be attributed to the started task. WLM will not double account. There is even benefit in defining a single transaction class for all APIs because it will allow you to distinguish between CPU consumed by the APIs (e.g data transformation) and CPU consumed by the server (e.g TLS and garbage collection processing).

  3. Hi Nigel,
    Can we achieve the load balancing/elasticity when the transaction can be configured to be executed in multiple CICS regions.

  4. Nigel Williams January 16, 2019

    Hello. Thanks for the question. The CICS service provider (SP) does not support request balancing across multiple CICS regions. However you can connect to a CICS region which then dynamically routes requests across multiple AORs using CICSPlex SM for example. Additionally you can use the IPIC HA support so that connections from the CICS SP are shared across multiple CICS regions. This is not request balancing but connection balancing. Connections remain established until a CICS region is recycled.

Join The Discussion

Your email address will not be published. Required fields are marked *