when Monitor model incoming events are deleted and it is going into some lock state occasionally, we get to see the below errors in JVM logs
[4/21/18 14:30:23:819 CDT] 0000006c XARminst E WTRN0037W: The transaction service encountered an error on an xa_recover operation. The resource was com.ibm.ws.rsadapter.spi.WSRdbXaResourceImpl@5728ed13. The error code was XAER_RMERR. The exception stack trace follows: javax.transaction.xa.XAException at oracle.jdbc.xa.OracleXAResource.recover(OracleXAResource.java:727) at com.ibm.ws.rsadapter.spi.WSRdbXaResourceImpl.recover(WSRdbXaResourceImpl.java:1111) at com.ibm.ws.Transaction.JTA.XARminst.recover(XARminst.java:141) at com.ibm.ws.Transaction.JTA.XARecoveryData.recover(XARecoveryData.java:719) at com.ibm.tx.jta.impl.PartnerLogTable.recover(PartnerLogTable.java:432) at com.ibm.tx.jta.impl.RecoveryManager.resync(RecoveryManager.java:1543) at com.ibm.tx.jta.impl.RecoveryManager.performResync(RecoveryManager.java:2276) at com.ibm.ws.tx.jta.RecoveryManager.performResync(RecoveryManager.java:119) at com.ibm.tx.jta.impl.RecoveryManager.run(RecoveryManager.java:2229) at java.lang.Thread.run(Thread.java:798)
[4/21/18 13:36:40:100 CDT] 00000251 EventPersiste E com.ibm.wbimonitor.router.persistence.withxct.EventPersistenceManagerImplOracle retrieveUnconsumedEventsCompleteUpdate() CWMRT6712E: An error occurred while retrieving an event. Schema: java.sql.SQLSyntaxErrorException: ORA-02049: timeout: distributed transaction waiting for lock
Table: {1}
Version: {2}
Queue ID: {3}
Caused by: {4}
[4/21/18 13:36:40:101 CDT] 00000251 ConsumerST_Qu E com.ibm.wbimonitor.mm.UHGEnrollments3.0.20180322011432.server.moderator.serialst.ConsumerST_QueueBypass retrieveMsgsFromQueue() CWMRT0040E: An error occurred while trying to close a JMS Queue Connection. Monitor will continue, but connections may be leaking. Monitoring Model ID: UHGEnrollments3.0 Monitoring Model Version: 20180322011432 Caused by: com.ibm.wbimonitor.server.moderator.exception.MessageSourceCloseException: com.ibm.wbimonitor.router.persistence.spi.EventPersistenceUpdateException: java.sql.SQLSyntaxErrorException: ORA-02049: timeout: distributed transaction waiting for lock Caused By: com.ibm.wbimonitor.router.persistence.spi.EventPersistenceUpdateException: java.sql.SQLSyntaxErrorException: ORA-02049: timeout: distributed transaction waiting for lock Caused By: java.sql.SQLSyntaxErrorException: ORA-02049: timeout: distributed transaction waiting for lock