The Streams console has new features that make it easier to detect memory leaks in your operators. In this example, I have introduced a memory leak into the
Below is the PE scatter chart very soon after the job was launched. The upper right corner indicates a PE that is using the most memory and the most CPU. This is the
If a Streams operator’s memory usage is constantly increasing as tuple arrives there is likely a memory leak occurring. Slower leaks are harder to detect using the graph over short periods of time but will be noticeable as applications run over night or for a couple of days. When we are creating Streams operators one of the tests we do is to run the operator over night with a steady flow of tuples to determine if the memory usage is increasing.
Other ways to monitor memory usage for leaks:
a) from the scatter graph hover on the element and watch the memory metric. While the hover is open this will update dynamically with every metric refresh cycle.
b) flip the graph over to view the grid and sort by memory. This can then be watched to determined if it is constantly rising.
The basic testing technique being employed is using a beacon to flood an operator with tuples to check for memory leaks in the operator’s process method. Since the beacon can generate very large volumes of tuples in a short period of time it is an excellent way to detect even small memory leaks.