Introduction

Dumps are a vital tool for application problems in production environments, allowing system administrators to capture comprehensive diagnostics information for later analysis by application developers and maintainers. For Node.js applications there are three main dump types available:

  1. node-report, a human-readable diagnostic summary, written to file. The report includes JavaScript and native stack traces, heap statistics, platform information and resource usage data.
  2. heap dump, a snapshot of the V8 heap in JSON format, providing a summary of the JavaScript objects in the heap, their types, sizes and references. You can use Chrome DevTools to inspect the heap dump.
  3. core dump, an operating system snapshot of the application’s process memory, written as a binary file. You can use the lldb debugger with the llnode plugin to inspect the core dump.

When running Node.js applications in the IBM Cloud Foundry environment it can be difficult to capture dumps because direct access to the application process is not readily available and because files written to the local application disk are lost when application containers are restarted. A new Node.js module, cloud-diagnostics makes capturing dumps on demand from Node.js applications in IBM Cloud Foundry easier. The module provides a consistent set of command-line and JavaScript APIs for generating the three Node.js dump types above, with support for dumps to be written to the IBM Cloud Object Storage persistence service for later collection and analysis. It uses the node-report, heap dump and gencore modules to provide the underlying dump support. This article describes how to use the cloud-diagnostics module to capture dumps from Node.js applications running in IBM Cloud Foundry.

This is one of three articles on how to capture dumps for Node.js applications running in IBM Cloud environments. Other articles in this series are:

Configuration

Add the cloud-diagnostics module to your application. You can do this by editing your application’s package.json file. Add the module to your application’s start command and dependencies:

  "scripts": {
    "start": "node -r cloud-diagnostics app.js"
  },
  "dependencies": {
    ...
    "cloud-diagnostics": "^1.0.0"
  },

You can also add the module into your application code using a require('cloud-diagnostics') statement. You will need to do this if you want to use the JavaScript APIs to generate dumps, see below.

Using the IBM Cloud Command Line Interface (CLI) to trigger dumps

The cloud-diagnostics module supports dump triggering via external signals sent to the Node.js process. The module uses signal handlers for the Linux real-time signals SIGRTMIN, SIGRTMIN+1, SIGRTMIN+2 to avoid interference with signals used by Node.js. These signals are handled by the cloud-diagnostics module and trigger a node report, heap dump or core dump respectively. The application will continue to run when the dump has completed. You can use the IBM Cloud CLI commands directly, or use the Linux script and Windows command files provided in the cloud-diagnostics module scripts directory. By default the cloud-diagnostics module will write dumps to the application container’s file system. You can configure the module to use persistent storage for the dumps so that they are preserved across container restarts, see ‘Using the IBM Cloud persistent Object Storage service for storing dumps’ below.

Log in to the IBM Cloud and Cloud Foundry CLI (using the ‘bx login’ and ‘cf login’ commands) then use the ‘cf ssh’ command to access the application container and send the required signal to the Node.js process. You can use the ‘cf apps’ command to list your applications and obtain the application names. To trigger a node report use the RTMIN signal, to trigger a heap dump use the RTMIN+1 signal, to trigger a core dump use the RTMIN+2 signal:

> cf ssh APPLICATION -c "pkill -RTMIN node"
> cf ssh APPLICATION -c "pkill -RTMIN+1 node"
> cf ssh APPLICATION -c "pkill -RTMIN+2 node"

You can also login to the application container shell using the ‘cf ssh APPLICATION’ command, then run the ‘pkill’ command from inside the shell. Alternatively use the ‘cfdump’ command provided in the cloud-diagnostics module scripts directory https://github.com/RuntimeTools/cloud-diagnostics/blob/master/scripts/unix/cfdump on Linux or https://github.com/RuntimeTools/cloud-diagnostics/blob/master/scripts/win/cfdump.cmd on Windows). The command takes two parameters. The first parameter allows you to specify whether you want a node report, heap dump or core dump, the second parameter is the name of your IBM Cloud Foundry application:

> cfdump node|heap|core APPLICATION

You will see messages on the IBM Cloud console log for your application indicating that the requested dump has been written:


If you are not using the IBM Cloud Object Storage service, dumps will be written to the application’s local disk. You can list and retrieve them using cf ssh commands:

> cf ssh APPLICATION -c "ls -l app"
> cf ssh APPLICATION -c "cat app/DUMP_FILE_NAME" > DUMP_FILE_NAME


Using the cloud-diagnostics module APIs to trigger dumps

The cloud-diagnostics module provides JavaScript APIs for writing node reports, heap dumps and core dumps. You can call these APIs to trigger dumps, for example on error paths within your application where detailed diagnostic data is required. By default these APIs will write dumps to the application container’s file system. You can configure the module to use persistent storage for the dumps so that they are preserved across container restarts, see ‘Using the IBM Cloud persistent Object Storage service for storing dumps’ below.

  // require the cloud-diagnostics module in the application
  const diagnostics = require('cloud-diagnostics');
 
  // trigger a node report
  diagnostics.storeNodeReport((err, filename) => {
    console.log('node report written to: ' + filename);
  });

  // trigger a heap dump
  diagnostics.storeHeapDump((err, filename) => {
    console.log('heap dump written to: ' + filename);
  });

  // trigger a core dump
  diagnostics.storeCoreDump((err, filename) => {
    console.log('core dump written to: ' + filename);
  });

Using the IBM Cloud persistent Object Storage service for storing dumps

The cloud-diagnostics module allows you to write dumps to persistent file storage – the IBM Cloud Object Storage service. The dumps will then be preserved across container restarts and be available for later collection and analysis. You can configure the Object Storage service using the IBM Cloud UI. First create an object storage service. The cloud-diagnostics module supports Object Storage OpenStack Swift for Bluemix, select this on the IBM Cloud UI:


Then connect the Object Storage service to your application:


Then open up the Object Storage service in the IBM Cloud UI and create a storage container using the Select Action -> Create Container pulldown. If you use “dumps” for the Object Storage container name, no further configuration is required. If you want to use a different name you will need to supply the name to the cloud-diagnostics module by providing a ‘cloud-diagnostics.json’ options file, see “Additional configuration options for the cloud-diagnostics module” below.


Then trigger a dump using the command-line interface or cloud-diagnostics module APIs as described above. You will see messages on the IBM Cloud console log for your application indicating that the requested dump has been written to the Object Storage service. You can use the IBM Cloud UI to list, download and delete dumps saved in the Object Storage service:


Additional configuration options for the cloud-diagnostics module

You can supply a cloud-diagnostics.json options file in your application’s home directory to control the behaviour of the cloud-diagnostics module. A sample options file is provided in the cloud-diagnostics module directory (node_modules/cloud-diagnostics). It contains the following default settings:

  {
    "name": "cloud-diagnostics",
    "nodereport": "api+signal",
    "heapdump": "api+signal",
    "coredump": "api+signal",
    "objectstorage": "dumps"
  }
  • The nodereport, heapdump and coredump properties control which triggers are enabled for each dump type. The default triggers for all three dump types are “api” and “signal”.
  • The objectstorage property sets the name of the storage container that will be used by the module when writing dump files to an Object Storage service (default is “dumps”).

Note that capture of dumps on failure (uncaught exception, fatal error, native code crash) is not currently supported by the cloud-diagnostics module. The Object Storage support runs JavaScript code to write the dumps, which is not possible when the Node.js application is failing. The cloud-diagnostics module is intended for use with running Node.js applications, with dumps triggered using signals or APIs as described above.

Join The Discussion

Your email address will not be published. Required fields are marked *