Apache Hadoop Insight Pack

Posted: September 29, 2015 Modified: October 9, 2015


Overview

The Apache Hadoop Insight Pack gives customers support to ingest data from various various Apache Hadoop logs into IBM Operations Analytics – Log Analysis. They can then use a rich set of analytical dashboards to visualize metadata and derived metrics from log content, along with performing additional queries on the content.

Apache Hadoop Insight Pack Health Dashboard

This dashboard displays charts for all the Apache Hadoop hosts. First two charts display all the hosts with log severity FATAL and ERROR over a configured time period. One chart displays the jobs failed for all the hosts configured time period. One chart also displays exception that occurred for all hosts.

Apache Hadoop Insight Pack Health Dashboard
Apache Hadoop Insight Pack Health Dashboard

Apache Hadoop Daemon Log Level Dashboard

This dashboard displays charts based on the log severity level. The first two charts depict the overall data related to Hadoop Cluster messages categorized by log severity level. Other two charts depict the overall data related to Hadoop Cluster categorized by daemon services log file path.

Apache Hadoop Insight Pack Daemon Log Level Dashboard
Apache Hadoop Insight Pack Daemon Log Level Dashboard

Apache Hadoop Insight Pack Job Status Dashboard

This dashboard displays charts based on the map reduce job data and memory utilization data of the container. Two charts display map reduce job data categorized by job status. Other two charts display five containers with minimum memory utilization.

Apache Hadoop Insight Pack Job Status Dashboard
Apache Hadoop Insight Pack Job Status Dashboard

Additional information

Apache Hadoop daemons (NameNode, SecondaryNameNode, DataNode, NodeManager and ResourceManager) generate logs. The logs include records related to normal daemon operation, as well as configuration logs, statistics, standard error, standard out, and internal diagnostic information. The Insight Pack includes support for ingesting and performing metadata searches against the default Apache Hadoop Daemon Logs.

Refer to the User Guide included in the Apache Hadoop Insight Pack for additional information on the Insight Packs and steps for installing and configuring.

Training Information

Review the following video for information on using and setting up the Insight Pack.


Overview presentation used in the training video.

View US Price & Buy

IBM Operations Analytics-Log Analysis Insight Packs for Infrastructure-Standard Install License.

Released: 29 September 2015
Version: 1.1.0.0
Cost: License Fee
Support: Supported

Resources

Product Information
Product Documentation
Forum

Requirements

  • IBM Operations Analytics – Log Analysis version 1.3.0 or above
  • Apache Hadoop 2.5.x
  • Apache Hadoop 2.6.0
  • Purchased “IBM Operations Analytics-Log Analysis Insight Packs for Infrastructure-Standard Install License” for each installed Insight Pack.

Support

This package is subject to the License terms included with the Insight Pack, along with those displayed upon download.

To report a problem with deploying this entry, entitled customers may contact the country specific IBM support channel, reference the IBM Worldwide Directory. Also use the “Support” link to access the support site for IBM Operations Analytics – Log Analysis information.