The Streams console has new features that make it easier to detect memory leaks in your operators. In this example, I have introduced a memory leak into the MatchRegex operator that allocates a structure on every tuple but never cleans up. This is a pathetic case to illustrate how to use the console to detect memory leak problems.

Below is the PE scatter chart very soon after the job was launched. The upper right corner indicates a PE that is using the most memory and the most CPU. This is the MatchRegex operator (hovering on that PE will show the name of the operators within it). At first glance this is fine since that operator is doing most of the analytic processing and it should be using the most memory as it keeps track of potential match sets. The PE is using less than 1MB.
memleak1

Here is the same graph 30 seconds (approximately) later. As you can see the amount of memory used is trending upward rapidly, almost 2.5MB.
memleak2

If a Streams operator’s memory usage is constantly increasing as tuple arrives there is likely a memory leak occurring. Slower leaks are harder to detect using the graph over short periods of time but will be noticeable as applications run over night or for a couple of days. When we are creating Streams operators one of the tests we do is to run the operator over night with a steady flow of tuples to determine if the memory usage is increasing.

Other ways to monitor memory usage for leaks:

a) from the scatter graph hover on the element and watch the memory metric. While the hover is open this will update dynamically with every metric refresh cycle.

b) flip the graph over to view the grid and sort by memory. This can then be watched to determined if it is constantly rising.

The basic testing technique being employed is using a beacon to flood an operator with tuples to check for memory leaks in the operator’s process method. Since the beacon can generate very large volumes of tuples in a short period of time it is an excellent way to detect even small memory leaks.

1 comment on"Streams Console: detecting operator memory leaks"

  1. Alexandr_Semeshchenko November 23, 2016

    Sorry, “The PE is using less than 1MB.”,”almost 2.5MB.” – what about from IBM Streams Knowledge Center >nMemoryConsumption Memory consumption that was used by the PE, in KB (resident, text, and data). ??? Byte or KB shows for nMemoryConsumption in your article ?

Join The Discussion