IIB and WMB write all message to the relevant system error log. On Unix platforms this is the syslog, on Windows this is the event viewer and on z/OS this is the job log. Each message output to the log has a unique “BIP number” corresponding to the cause of the message. Messages have 3 severity levels:
I = Informational These messages are for Informational purposes and do not require further action, for example Informational messages are issued when a Broker starts, when a connection is made to a JMS provider etc.
W=Warning These messages are are often used to draw attention to conditions that while not errors may be unintended and require action to reconfigure the affected flows.
Each BIP message also gives the name of the Broker, the process ID and the thread ID and EG name where the problem occurred in order to make it easier to group errors for particular workloads.
All BIP messages have a recommended “User Response” section which indicates how the error may be resolved however due to limitations in the syslog technology this is only visible by default on Windows platforms. In order to see the user response for a particular BIP message you can use the command:
When a node in a message flow throws an exception this is assigned a BIP number. As the exception is propagated up the flow, possibly being processed by various exception handling routines attached to the catch terminal of nodes it may be “wrapped” in several outer exceptions. This forms a “stack” of exceptions known as an ExceptionList (see the section on the ExceptionList below).
If an exception reaches an input node without being handled by any branch of the flow then the input transaction is rolled back and each message in the ExceptionList is output to the syslog. On IIB v9 the position within a nested stack of exceptions is indicated by the “(Msg 1/4)” decorators.
For many transports if the root cause (usually the innermost exception of the ExceptionList) is transient a retry mechanism will mean that these errors will be resolved without requiring intervention from a user. For instance an MQInput node will retry the message until the “backout threshold” is reached at which point it will requeue the message to the back out queue or dead letter queue.
Where errors are a result of message processing at least one message in the exception stack will usually identify the message flow that is causing the problem.
BIP messages as a result of message processing are therefore most often of interest to message flow developers who understand the flow logic.
As well as messages logged to the syslog as a result of message processing there will also be messages emitted corresponding to Administrative actions. These messages could be due to administrative actions such as deployment or configuration of a broker component. They could also be reports about the state of the broker itself. For instance if the Broker crashes unexpectedly then a message is written to inform the Administrator there is a problem before restarting the process.
Errors that are the result of administrative actions will usually need action from a Broker Administrator in order to resolve.
Exception List
| Problem Type |
Message Processing |
| Environment Type |
All |
| Applicable Versions |
All |
WebSphere Message Broker controls error behaviour through the use of exceptions. When a node encounters a problem an exception is thrown and this may be caught and handled by any node which has a Catch terminal wired in the upstream flow. When an exception is caught but can not be handled by a node it is generally wrapped in another exception which adds context information and then thrown upstream.
This creates a nested “stack” of exceptions where typically the innermost exception contains the root cause of the problem and the outer exceptions provide increasingly general information about the path of the exception through the flow.
Most nodes in WMB do not have access to the exception objects directly (apart from .NETCompute and JavaCompute nodes) so instead the broker provides a way to process exceptions which is a natural extension of the existing transformation capabilities. When execution is propagated from either a catch terminal or a failure terminal then a new message tree is added to the message assembly called “ExceptionList”. This can be navigated like any other tree in the product using any of the usual transformation capabilities.
Additionally each exception is mapped onto a “BIP” error message, this serves two main purposes:
1.) If the exception reaches an input node and still has not been handled then the entire exception stack is output to the Broker Error log. The individual exceptions are serialized as BIP messages to the Broker Error Log as describe above.
2.) The BIP message provides structure when building the ExceptionList message tree
Lets examine an example. In the following scenario we have an error thrown by an MQ Input Node because the queueName parameter does not exist. The catch terminal has been wired and a trace node used to display the contents of the ExceptionList:
In this case the output from the trace node looks like this:
( [‘MQROOT’ : 0x7fc950050090]
(0x01000000:Name):RecoverableException = (
(0x03000000:NameValue):File = ‘/build/slot1/S900_P/src/DataFlowEngine/MQLibrary/linklib/ImbMqOutputNode.cpp’ (CHARACTER)
(0x03000000:NameValue):Line = 877 (INTEGER)
(0x03000000:NameValue):Function = ‘ImbMqOutputNode::evaluate’ (CHARACTER)
(0x03000000:NameValue):Type = ‘ComIbmMQOutputNode’ (CHARACTER)
(0x03000000:NameValue):Name = ‘failingMQFlow#FCMComposite_1_2′ (CHARACTER)
(0x03000000:NameValue):Label = ‘failingMQFlow.MQ Output’ (CHARACTER)
(0x03000000:NameValue):Catalog = ‘BIPmsgs’ (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 2230 (INTEGER)
(0x03000000:NameValue):Text = ‘Caught exception and rethrowing’ (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 14 (INTEGER)
(0x03000000:NameValue):Text = ‘failingMQFlow.MQ Output’ (CHARACTER)
)
(0x01000000:Name ):MessageException = (
(0x03000000:NameValue):File = ‘/build/slot1/S900_P/src/DataFlowEngine/MQLibrary/linklib/ImbMqOutputNode.cpp’ (CHARACTER)
(0x03000000:NameValue):Line = 1989 (INTEGER)
(0x03000000:NameValue):Function = ‘ImbMqOutputNode::putMessage‘ (CHARACTER)
(0x03000000:NameValue):Type = ‘ComIbmMQOutputNode’ (CHARACTER)
(0x03000000:NameValue):Name = ‘failingMQFlow#FCMComposite_1_2′ (CHARACTER)
(0x03000000:NameValue):Label = ‘failingMQFlow.MQ Output’ (CHARACTER)
(0x03000000:NameValue):Catalog = ‘BIPmsgs’ (CHARACTER)
(0x03000000:NameValue):Severity = 3 (INTEGER)
(0x03000000:NameValue):Number = 2666 (INTEGER)
(0x03000000:NameValue):Text = ‘Failed to open queue’ (CHARACTER)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 2 (INTEGER)
(0x03000000:NameValue):Text = ‘-1’ (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = ‘MQW101’ (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 2 (INTEGER)
(0x03000000:NameValue):Text = ‘2085’ (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = ” (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = ” (CHARACTER)
)
(0x01000000:Name ):Insert = (
(0x03000000:NameValue):Type = 5 (INTEGER)
(0x03000000:NameValue):Text = ‘IDONTEXIST’ (CHARACTER)
)
)
)
)
Here we can see there are 2 nested exception in the following structure:
BIP2230
->BIP2666
The inner most message contains the inserts relating to the MQ error whereas the outer message provides context indicating that the MQOutput Node was unable to handle the exception and allowed it to be rethrown up the flow. No other node in the flow attempted to handle the exception so we see no other entires in the exception stack.
If we unwired the catch terminal and allowed the message to rollback we would see the exception list output to the Broker Event Log as shown below:
May 20 15:57:21 prionson IIB[6139]: IBM Integration Bus v9001 (BRK9.default) [Thread 14702] (Msg 1/3) BIP2628E: Exception condition detected on input node ‘failingMQFlow.MQ Input’.
May 20 15:57:21 prionson IIB[6139]: IBM Integration Bus v9001 (BRK9.default) [Thread 14702] (Msg 2/3) BIP2230E: Error detected whilst processing a message in node ‘failingMQFlow.MQ Output’.
May 20 15:57:21 prionson IIB[6139]: IBM Integration Bus v9001 (BRK9.default) [Thread 14702] (Msg 3/3) BIP2666E: An error occurred in node ‘failingMQFlow.MQ Output’ when opening queue ‘IDONTEXIST’ on queue manager ”. State = ‘-1’ ‘MQW101’ ‘2085’ ”
May 20 15:57:22 prionson IIB[6139]: IBM Integration Bus v9001 (BRK9.default) [Thread 14702] (Msg 1/1) BIP2648E: Message backed out to a queue; node ‘failingMQFlow.MQ Input’.
The first three entries correspond to the ExceptionList in the trace node example, here the message text has been added in the appropriate language for the locale and the inserts copied in to make the error human readable. The MQInput Node has also wrapped the exception list in another BIP2628 exception indicating that the MQInput node caught the exception and made some effort to process it (it checked if the catch terminal was wired for example in case the user wanted to process the exception in the flow). The “(Msg X/Y)” indicator was added at IIB v9 and shows the relative position of messages within the ExceptionList. The MQInput node has also output a separate BIP message, not part of the exception list to say that a message was backed out because it couldn’t be processed.
The main benefit of the ExceptionList structure is that you can work with failures within the flow to either provide a unified reporting framework or to handle particular error with changes in application logic. For example the following ESQL snippet walks the exception list using standard tree navigation and flattens the ExceptionList into an XMLNSC message containing an element per exception:
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
CALL CopyMessageHeaders();
CREATE LASTCHILD OF OutputRoot DOMAIN(‘XMLNSC’);
SET OutputRoot.XMLNSC.(XMLNSC.XmlDeclaration)*.(XMLNSC.Attribute)Version = ‘1.0’;
SET OutputRoot.XMLNSC.(XMLNSC.XmlDeclaration)*.(XMLNSC.Attribute)Encoding = ‘UTF-8’;
CREATE LASTCHILD OF OutputRoot.XMLNSC NAME(‘ERRORMESSAGE’);
DECLARE outputRef REFERENCE TO OutputRoot.XMLNSC.ERRORMESSAGE;
DECLARE exceptionRef REFERENCE TO InputExceptionList.*[1];
WHILE LASTMOVE(exceptionRef) DO
IF exceptionRef.Number IS NOT NULL THEN
CREATE LASTCHILD OF outputRef NAME(‘ERROR’) VALUE (‘ERRORNUM: ‘ ||
CAST(exceptionRef.Number AS CHARACTER) || ‘ TEXT:’ ||
exceptionRef.Text);
END IF;
MOVE exceptionRef LASTCHILD;
END WHILE;
RETURN TRUE;
END;
This produces output like the following:
ERRORNUM: 2230 TEXT:Caught exception and rethrowing
ERRORNUM: 7499 TEXT:CLAIMSPROCESSING FAULT. Code: ERR004
ErrorText: Could not view claim. Reason: Claim CLAIM101
does not exist.
Standard Output and Standard Error
| Problem Type |
Third-party libraries, Java, JCN Code |
| Environment Type |
Development, Test, Production |
| Applicable Versions |
All |
All applications have a stdout and stderr stream associated with them. The output of these streams are what you see when you run a typical application from the console. Message Broker is designed to run as a non-interactive task and so its stdout and stderr streams are not automatically output to the command console.
Instead these streams are redirected to files on the file system. On Windows systems the streams are re-directed to:
Broker
%MQSI_WORKPATH%\components\\console.txt (contains both stdout and stderr)
Execution Group
%MQSI_WORKPATH%\components\\\console.txt (contains both stdout and stderr)
On Unix Systems the streams are re-directed to:
Broker
$MQSI_WORKPATH/components//stdout
$MQSI_WORKPATH/components//stderr
ExecutionGroup
$MQSI_WORKPATH/components///stdout
$MQSI_WORKPATH/components///stderr
On zOS the stdout and stderr streams are appended to the JOBLOG.
Typically WMB tries to avoid relying on the stdout and stderr streams for debugging information, however if you are using any third party libraries with Message Broker or if Message Flow developers use System.out.println statements in Java Compute Nodes then debugging information can end up in these files. It is also common for Third-party Java code to call the printStackTrace() function when catching errors.
For these reasons it is sometimes useful to check the stdout and stderr streams when troubleshooting as a failure in a Java Compute Node or a Third-party library may not be properly logged by the Message Flow developer.
Note that these files are not rotated or otherwise managed so they will continue to grow until they fill all available file system space. If you ened to clear out these files it is safe to delete them when the Broker is stopped. The Broker will automatically recreate the files again on startup.