CPU usage high in CICS region with tasks in JVMTHRED waits

A CICS system dump at the time of high CPU was taken. It looks like there was not enough CPU for 45 concurrent threads in the JVM. The increase in workload then backed up into JVMTHRED waits which isn't a good wait to be in, ultimately resulting in a situation that is difficult to recover from.

The problem with many tasks being in a JVMTHRED wait is that when a thread is freed DFHSJTH resumes all waiters, not just the one that has been waiting the longest. The first of these waiters to get dispatched on the QR will secure the thread and be dispatched into the JVM. All the other tasks will run briefly on QR only to be put back into a JVMTHRED wait.

The dumps show many of the tasks were suspended hundreds of times attempting to get a thread. This all consumes QR unnecessarily which doesn't help the state of the region.

Increasing the thread limit of the JVMSERVER will likely not help. Instead, IBM recommends doing workload testing to determine how many concurrent tasks can be run in a JVMSERVER before the response time exceeds what is acceptable. Then set the thread limit and TCLASS to this number (or slightly less). With this configuration the tasks will queue up in the TCLASS wait and once resumed be able to get a thread immediately. Tasks waiting in a TCLASS wait are resumed in order and one at a time, making it a much better wait than JVMTHRED.

Note that assumes there are no other transactions that run in this JVMSERVER. If there are other transactions, then you will want to take this into account when setting the thread limit for the JVM server (THREADLIMIT) and TCLASS to ensure they have the opportunity to run as appropriate.