Memory management issues are very common when using Java oriented middleware. IBM BigInsights contains various open source projects which utilize the java virtual machine as its underlying infrastructure, open source projects such as HDFS, MapReduce, Console, Hbase, Hive, Flume, JAQL, Yarn, Spark etc.. .

When dealing with these open source projects it is often useful to understand not only the open source project ecosystem itself but the JDK and memory management internals which facilitates the activities of these projects. Understanding these internals will enable the user to make sure the open source code will run as efficiently as possible on the IBM’s java platform.

In this article we will describe how will we go about generating garbage collection output (verbosegc) for several components in order to analyze the performance of the garbage collector, and we will use IBM’s support assistant in order to diagnose the data we have received from the java garbage collector.

In order to get started with article you should have the following installed on your machine

  • IBM BigInsights quick start edition ,which can be downloaded from : http://www-01.ibm.com/software/data/infosphere/biginsights/quick-start/downloads.html ,If you have a running BigInsights cluster then you are already set and there is no need to download the quick start edition.
  • IBM support assistant workbench ,which can be downloaded from: http://www-01.ibm.com/software/support/isa/workbench.html.
  • IBM pattern modeling and analysis tool for java garbage collection plug-in installed on the “ISA workbench”.

Verbose GC output analysis
Image1: Memory analysis on the IBM pattern modeling and analysis tool for java garbage collection plug-in

We will select 3 sample open source projects and work with them in order to analyze the data gathered from the garbage collector. These projects/extensions are:

  • Hbase
  • Console
  • Map Reduce

Download the complete article here :
Dealing with memory issues with IBM Biginsights and IBM JDK

Join The Discussion

Your email address will not be published. Required fields are marked *