In this post, I’ll discuss how our Simple Data Pipe sample app uses the Bunyan Node.js logging framework to capture detailed logging information about a pipe run. Then I’ll show you how to analyze the report using the Bunyan viewer tool.
If you’ve explored our Simple Data Pipe tutorial on Bluemix, you know that metadata about your pipe runs is stored in Cloudant as JSON. Cloudant’s support for binary attachments within JSON lets you attach the logs from Bunyan right alongside their associated JSON document, which you can then access for further analysis.
A word about Bunyan
Bunyan is a simple and fast JSON logging library for Node.js services. It can be configured to output the data to streams that can be stored anywhere. Simple Data Pipe uses this library to capture log information about a particular run, then attach the report to the pipe run document stored in the pipe_db database in your Cloudant account.
This logging framework supports many log levels:
error. As you’ll see, Bunyan also provides a CLI utility to pretty-print its output, with the ability to filter by logging group and level. See https://github.com/trentm/node-bunyan for more information.
How to locate the log for a particular run
Here’s the scenario: You attempted a pipe run and something went wrong. You now need to locate the log, download it from the pipe_db Cloudant database, and analyze it for troubleshooting.
- Go to Bluemix and click on your pipe app instance.
- Click on the pipes-cloudant-service box to open the Cloudant dashboard.
- Click the Launch button.
- In the Cloudant dashboard, click on the pipe_db database.
- In the menu on the left, click _design/application, then Views, then all_runs.
- Locate your last run.
On the left-hand side of the all_runs view, you can see that the Map function logic that defines this view indexes pipe run documents sorted in chronological order. So, the run document you’re looking for is the last one in the view. (You may need to page through the results a few times if you have performed a lot of runs.)
- Click on the pencil icon to open the run document. You should be able to see the JSON metadata for the run.
- In the toolbar above the document, click the View Attachments dropdown button and right-click on run.log. Then click on the Save Link As… option. This will download the file to a directory of your choice.
- The next step will be to use the Bunyan CLI tool to analyze run.log.
If you have not already done so, install Bunyan on your local machine using these simple steps:
npm install bunyan -g
Note: If you are on a Linux-based system like Mac OS X, use
You can view the entire log in pretty-printed format using this command:
Note: To quit long output, use
You can also filter the log to only view errors using the following command:
bunyan <path/to/run.log> -l error
Note: You can use any log level you want, e.g.,