Kubernetes with OpenShift World Tour: Get hands-on experience and build applications fast! Find a workshop!

Monitor and correlate personal asthma readings with environmental data, Part 2

If you have been following along with Part 1, you are half-way through building a web-based PHP application to store and analyze personal peak flow meter asthma readings. In the first part, you created the application skeleton, added an input form for meter readings, and saved those readings to a Cloudant database running on the Cloud. In this concluding part, you integrate weather data with the readings, create graphs from the collected data for easier analysis, and deploy the result on the cloud.

Learning objectives

The example application shown in this tutorial allows users to input and save peak flow meter readings into a database. Each time a reading is added, the application automatically retrieves current external weather conditions, such as temperature and humidity, and enhances the reading with this additional metadata. The users can see visualizations and charts of the saved data, such as a bar chart of peak flow averages grouped by month, or a scatter plot of peak flow values versus temperature.

Behind the scenes, the application uses the IBM Cloud Cloudant service to save data to a Cloudant database in the cloud and the IBM Cloud Weather Company service to retrieve current weather conditions. It also uses the Slim PHP micro-framework to process requests, Bootstrap to create a mobile-optimized user experience, and Chart.js to create different types of charts and graphs from the saved data.

After completing this second part of the tutorial series, you can complete the following tasks:

  • Integrate data from Weather Company’s Data API with your IBM Cloud application.
  • Create different types of HTML5 charts to visualize data.
  • Deploy your final application code on IBM Cloud and debug any runtime errors.

Although this IBM Cloud application uses PHP with the Cloudant and Weather Company Data services, you can apply similar logic to other languages and other IBM Cloud services.

The complete source code for this application is available on GitHub at /vvaswani/bluemix-asthma-tracker/.

Prerequisites

Before starting, make sure you have the following tools and skills:

NOTE: Any application that uses the Cloudant service and Weather Company Data service on IBM Cloud must comply with the corresponding Terms of Service at IBM Cloudant for IBM Cloud (11-2018) and Weather Company Data for IBM Cloud (10-2018). Similarly, any application that uses the IBM Cloud must comply with their respective terms of use, described on the IBM Cloud. Before beginning your project, spend a few minutes reading these terms and ensuring that your application complies with them.

Estimated time

This tutorial consists of two parts. It should take you approximately 60 minutes to complete each part.

Steps

In this second part of this tutorial series, you complete the following steps:

  1. Start the Weather Company Data service.
  2. Integrate current weather data with meter readings.
  3. Retrieve, display and delete meter readings.
  4. Display a bar chart of monthly averages.
  5. Create a scatter chart of meter readings versus humidity.
  6. Deploy the application on IBM Cloud.

Step 1: Start the Weather Company Data service

One of the goals of the application is to make it easier to correlate asthma changes with seasonal or weather changes. In the first part of this series, you wrote the code to save meter readings to a database. Now it’s time to enhance those readings with additional weather metadata. The Weather Company Data service on IBM Cloud provides an easy way to get this information.

  1. To create the service, log in to your IBM Cloud account and, from the service catalog, under Analytics, select the Weather Company Data service. Review the description of the service, select the Free Plan and click Create.

    A new Weather Company Data service is initialized, and the service information page opens.

  2. Under Service credentials, click New credential and then click Add to confirm. You can view the credentials for the instance by clicking the View credentials button.

  3. Copy the URL from the JSON credentials block into your application’s configuration file at $APP_ROOT/config.php, as shown in the following example. Add the latitude and longitude of your current location (you can obtain this information from a tool like Google Maps).

     <?php
     // config.php
     $config = [
       'settings' => [
         'displayErrorDetails' => true,
         // other configuration //
         'twc' => [
           'uri' => 'TWC-URI',
           'latitude' => 'LATITUDE',
           'longitude' => 'LONGITUDE'
         ]
       ]
     ];
    

As of this writing, the free Weather Company Data service plan allows a maximum of 10 calls to service per minute, with a maximum of 10,000 calls total for your user.

Step 2: Integrate current weather data with meter readings

To see the Weather Company Data service in action, open RESTer in your browser and send a GET request to the URL endpoint: https://twcservice.mybluemix.net/api/weather/v1/geocode/LATITUDE/LONGITUDE/observations.json?units=m.

Replace LATITUDE and LONGITUDE with valid coordinates, and when prompted, enter the user name and password that you obtained in Step 1. The service should return the current weather conditions for the specified coordinates.

Here’s an example of a response to the https://twcservice.mybluemix.net/api/weather/v1/geocode/51.5074/0.1278/observations.json?units=m request for the current weather in London:

Screen capture

In particular, note the temp and rh keys of the response, which indicate the current temperature and humidity respectively.

Several other URL endpoints are also available in the Weather Company Data service. You can find out more about them in the service documentation.

Now that you know how to retrieve current weather data, you need to integrate this it into the application. Complete the following steps:

  1. Begin by updating the application initialization script at $APP_ROOT/public/index.php and initialize a new instance of the Guzzle HTTP client to handle weather API requests:

     <?php
     // index.php
     // ...
     // configure and add Weather Company API client to DI container
     $container['twc'] = function ($container) use ($config) {
       return new Client([
         'base_uri' => $config['settings']['twc']['uri'] . '/api/weather/v1/',
         'timeout'  => 6000,
         'verify' => false,    // set to true in production
       ]);
     };
     // ...
     $app->run();
    
  2. Update the callback function for the /save route to request current weather conditions using the client, extract the temperature and humidity from the response, and add it to the saved reading in Cloudant. See the following revised code for the callback function, with the additions highlighted:

      <?php
      // index.php
      // ...
    
      // process form
      $app->post('/save', function (Request $request, Response $response, $args) {
        $config = $this->get('settings');
        $params = $request->getParams();
    
        // validate input
        $value = filter_var($params['value'], FILTER_SANITIZE_NUMBER_INT);
        if (!(filter_var($value, FILTER_VALIDATE_INT))) {
          throw new Exception('ERROR: Value is not valid');
        }
        $note = filter_var($params['note'], FILTER_SANITIZE_STRING);
    
        **// request weather data from TWC API
        // extract temperature and humidity from response
        $twcResponse = $this->twc->get('geocode/' . $config['twc']['latitude'] . '/' . $config['twc']['longitude'] . '/observations.json?units=m');
        $twcData = json_decode($twcResponse->getBody());
        $temp = $twcData->observation->temp;
        $rh = $twcData->observation->rh;**
    
        // create document with reading, time and weather data
        $doc = [
          'value' => $value,
          'note' => $note,
          **'temp' => $temp,**
          **'rh' => $rh,**
          'ts' => mktime()
        ];
    
        // save document to Cloudant
        $cloudantResponse = $this->cloudant->post($config['db']['name'], [
          'json' => $doc
        ]);
        if ($cloudantResponse->getStatusCode() == 201) {
          return $response->withHeader('Location', $this->router->pathFor('list'));
        } else {
          throw new Exception('ERROR: Document could not be created');
        }
      })->setName('save');
    
      // ...
      $app->run();
    

With this change, every document that is saved to the Cloudant database additionally includes the temperature and humidity for the specified location. The following screen capture shows an example of an entry:

Screen capture of a temperature and humidity entry

Step 3: Retrieve, display and delete meter readings

Complete the following steps to update the application to show a list of all the readings in the database:

  1. Add a new /list route and callback function that performs a GET request to Cloudant to retrieve all available documents, as in the following example:

     <?php
     // index.php
     // ...
    
     // list page controller
     $app->get('/list', function (Request $request, Response $response) {
       $config = $this->get('settings');
    
       // get all docs in database
       // include all content
       $dbResponse = $this->cloudant->get($config['db']['name'] . '/_all_docs', [
         'query' => ['include_docs' => 'true', 'descending' => 'true']
       ]);
       if($response->getStatusCode() == 200) {
         $json = (string)$dbResponse->getBody();
         $body = json_decode($json);
       }
       // provide results to template
       $response = $this->view->render($response, 'list.twig', [
         'router' => $this->router, 'results' => $body->rows
       ]);
       return $response;
     })->setName('list');
    
     // ...
     $app->run();
    

    The list of documents is transferred to the template in the $results array.

  2. Create the corresponding template at $APP_ROOT/views/list.twig to iterate over the array and display each entry, as shown in the following example:

     {% extends 'home.twig' %}
    
     {% block content %}
     <h3 class="display-6">List of peak flow readings</h3>
     {% if results|length > 0 %}
     <table class="table">
       <thead>
         <tr>
           <th scope="col">Date/time</th>
           <th scope="col">Peak flow</th>
           <th scope="col">Temperature</th>
           <th scope="col">Humidity</th>
           <th scope="col"></th>
         </tr>
       </thead>
       <tbody>
       {% for result in results %}
         <tr>
           <td>{{ result.doc.ts|date("d M Y H:i") }}</td>
           <td>{{ result.doc.value }} L/min</td>
           <td>{{ result.doc.temp }} °C</td>
           <td>{{ result.doc.rh }}%</td>
           <td><a href="{{ path_for('delete', {'id':result.doc._id, 'rev':result.doc._rev}) }}" class="btn btn-success">Delete</a></td>
         </tr>
       {% endfor %}
       </tbody>
     </table>
     Attribution: This application uses data from the Weather Company Data service on IBM Cloud. All data, logos and trademarks are copyright their respective owners.
     {% else %}
     No data found.
     {% endif %}
     {% endblock %}
    
  3. To see the output, browse to http://localhost/list. Here’s an example of what you should see:

    Screen capture of list of peak flow readings

    Pay attention to the following two points:

    • Although each reading has a timestamp, the readings are not automatically sorted by date. This is normal behavior with Cloudant. To sort by date, you must create a view with the timestamp as key and retrieve documents using that view. Learn more about this in the CouchDB documentation on views.

    • Each reading has a Delete button, which points to the /delete route and includes the document identifier and revision identifier in its hyperlink. Document deletion requests sent to Cloudant must include both the document identifier and revision identifier.

  4. Activate the Delete buttons by implementing the /delete route and callback handler in $APP_ROOT/public/index.php, as shown in the following example:

     <?php
     // index.php
     // ...
    
     // deletion handler
     $app->get('/delete/{id}/{rev}', function (Request $request, Response $response, $args) {
       $config = $this->get('settings');
    
       // get document ID and revision ID in Cloudant
       $id = $args['id'];
       $rev = $args['rev'];
    
       // delete document from database using Cloudant API
       $this->cloudant->delete($config['db']['name'] . '/' . $id, [
         "query" => ["rev" => $rev]
       ]);
    
       return $response->withHeader('Location', $this->router->pathFor('list'));
     })->setName('delete');
    
     // ...
     $app->run();
    

    This handler receives the document and revision identifiers as URL parameters and generates a DELETE request using the Cloudant API client. In response, Cloudant deletes the corresponding document from the database.

Step 4: Display a bar chart of monthly averages

Now that you’re able to retrieve data from Cloudant, you need to convert it from a mass of numbers to visual charts and graphs, which make analysis easier. A good tool for this purpose is Chart.js, a client-side JavaScript library that makes it easy to generate HTML5 charts. Chart.js is available under the MIT license and comes with detailed documentation.

A common question is, “How does my asthma change from season to season?” To answer this question, compare average peak flow meter readings across months by generating a bar chart with Chart.js from the data in the Cloudant database.

  1. Begin by adding the following callback function to $APP_ROOT/public/index.php:

     <?php
     // index.php
     // ...
    
     // bar chart controller
     $app->get('/bar', function (Request $request, Response $response, $args) {
       $config = $this->get('settings');
    
       // get all docs in database
       // include all content
       $dbResponse = $this->cloudant->get($config['db']['name'] . '/_all_docs', [
         'query' => ['include_docs' => 'true']
       ]);
       if($response->getStatusCode() == 200) {
         $json = (string)$dbResponse->getBody();
         $body = json_decode($json);
       }
    
       // iterate over observations
       // build a monthly histogram
       foreach ($body->rows as $row) {
         $month = date('m', $row->doc->ts);
         $value = $row->doc->value;
         $observations[$month][] = $value;
       }
    
       // iterate over collected observations
       // calculate monthly average
       foreach ($observations as $key => $value) {
         $sum = array_sum($value);
         $average = round($sum / count($value));
         $results[$key] = $average;
       }
    
       // provide results to template
       $response = $this->view->render($response, 'bar.twig', [
         'router' => $this->router, 'results' => $results
       ]);
       return $response;
     })->setName('bar');
    
     // ...
     $app->run();
    

    The first few lines of this callback handler do the same thing as those of the /list handler shown previously. They connect to Cloudant and retrieve all the documents available in the database. The handler then iterates over the result set, checking the timestamp value of each reading and sorting them into 12 “buckets”, one for each month. After all the documents are sorted in this manner, the handler iterates over the buckets, calculating the average reading for each month. The final list of monthly averages is then transferred to the results page template.

  2. To complete the feature, create a page template at $APP_ROOT/views/bar.twig with the following code:

      {% extends 'home.twig' %}
    
     {% block links %}
     <a class="btn btn-primary" href="{{ path_for('list') }}">List</a></button>
     {% endblock %}
    
     {% block content %}
     <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.7.3/Chart.bundle.js" crossorigin="anonymous"></script>
     <h3 class="display-6">Bar chart of average monthly peak flow readings</h3>
     <canvas id="myChart" width="400" height="400"></canvas>
    
     {% if results|length > 0 %}
     <script>
     var ctx = document.getElementById("myChart");
     var scatterChart = new Chart(ctx, {
       type: 'bar',
       data: {
         labels: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"],
         datasets: [{
           data: [
           {% for i in 1..12 %}
             {{ results['%02d'|format(i)] }},
           {% endfor %}
           ],
           backgroundColor: 'rgb(255, 0, 0)',
         }],
       },
       options: {
         legend: {
           display: false
         },
         scales: {
           yAxes: [{
             scaleLabel: {
               display: 'true',
               labelString: 'Average PF (L/min)'
             },
             ticks: {
                 suggestedMin: 200,
                 suggestedMax: 700
             }
           }],
         }
       }
     });
     </script>
     {% else %}
     No data found.
     {% endif %}
     {% endblock %}
    

This template first initializes the Chart.js library and creates a <canvas> HTML element to display the chart. It then initializes an empty bar chart, defines the X axis scale as the months of the year, and iterates over the list of monthly averages to set the corresponding Y axis value for each month. Refer to the Chart.js bar chart documentation for a complete explanation of the parameters used to generate the chart.

To see the result, browse to http://localhost/bar. You should see something like the following image (obviously, your results will vary depending on the number and value of meter readings in your database).

Screen capture of asthma diary bar chart

Step 5: Create a scatter chart of meter readings versus humidity

Another useful bit of information is knowing how humidity affects your asthma – for example, if high humidity correlates with greater difficulty in breathing. With Chart.js, it’s easy to produce a scatter chart that displays this correlation (or lack thereof).

  1. Begin by adding the following callback function to $APP_ROOT/public/index.php:

     <?php
     // index.php
     // ...
    
     // scatter chart controller
     $app->get('/scatter/humidity', function (Request $request, Response $response, $args) {
       $config = $this->get('settings');
    
       // get all docs in database
       // include all content
       $dbResponse = $this->cloudant->get($config['db']['name'] . '/_all_docs', [
         'query' => ['include_docs' => 'true']
       ]);
       if($response->getStatusCode() == 200) {
         $json = (string)$dbResponse->getBody();
         $body = json_decode($json);
       }
    
       // provide results to template
       // specify metric to use depending on type of chart requested
       $response = $this->view->render($response, 'scatter.twig', [
         'router' => $this->router, 'results' => $body->rows,
         'metric' => ['rh', 'Humidity (%)']
       ]);
       return $response;
     })->setName('scatter');
    
     // ...
     $app->run();
    

    The first few lines of this callback handler connect to Cloudant and retrieve all the documents available in the database. The handler then transfers all the documents to the results page template, together with a variable indicating the metric to use and the human-readable label for the metric (for display on the chart axis).

  2. To turn the readings into a scatter chart, create a page template at $APP_ROOT/views/scatter.twig with the following code:

     {% extends 'home.twig' %}
    
     {% block links %}
     <a class="btn btn-primary" href="{{ path_for('list') }}">List</a></button>
     {% endblock %}
    
     {% block content %}
     <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.7.3/Chart.bundle.js" crossorigin="anonymous"></script>
     <h3 class="display-6">Scatter chart of peak flow readings</h3>
     <canvas id="myChart" width="400" height="400"></canvas>
    
     {% if results|length > 0 %}
     <script>
     var ctx = document.getElementById("myChart");
     var scatterChart = new Chart(ctx, {
       type: 'scatter',
       data: {
         datasets: [{
           pointBackgroundColor: 'rgb(255, 0, 0)',
           data: [
           {% for result in results %}
           {
               x: {{ result.doc.value }},
               y: {{ attribute(result.doc, metric[0]) }}
           },
           {% endfor %}
           ]
         }]
       },
       options: {
         legend: {
           display: false
         },
         scales: {
           xAxes: [{
             scaleLabel: {
               display: 'true',
               labelString: 'PF (L/min)'
             },
             type: 'linear',
             position: 'bottom',
             ticks: {
                 suggestedMin: 100,
                 suggestedMax: 700
             }
           }],
           yAxes: [{
             scaleLabel: {
               display: 'true',
               labelString: '{{ metric[1] }}'
             },
             type: 'linear',
             position: 'left',
             ticks: {
                 suggestedMin: 20,
                 suggestedMax: 45
             }
           }],
         }
       }
     });
     </script>
     {% else %}
     No data found.
     {% endif %}
     {% endblock %}
    

    This template initializes the Chart.js library and then iterates over the documents provided to create a data set of peak flow readings and corresponding humidity readings. It then plots these values on a scatter chart. See the Chart.js scatter chart documentation for a complete explanation of the parameters used to generate the chart.

    To see the result, go to http://localhost/scatter/humidity. You should see something like the following image (although your results may vary depending on the number of meter readings in your database).

Screen capture of asthma diary scatter chart

In a similar vein, it is also possible to plot a scatter chart of meter readings versus temperature (or any other weather metric you wish to use). The source code repository for this tutorial contains a variant of the previous callback handler showing the necessary code.

Step 6: Deploy the application on IBM Cloud

To connect to the Cloudant and Weather Company Data services, the PHP application needs the URL and access credentials. You have previously specified this information in the application configuration file. However, as an alternative, you can bind the Cloudant service to the application and import its credentials directly from the IBM Cloud environment (the VCAP_SERVICES variable).

  1. To use this approach, add the following code to the $APP_ROOT/public/index.php script, before the lines that initialize the database connection:

     <?php
     // index.php
     // ...
    
     // if VCAP_SERVICES environment available
     // overwrite local credentials with environment credentials
     if ($services = getenv("VCAP_SERVICES")) {
       $services_json = json_decode($services, true);
       $config['settings']['db']['uri'] = $services_json['cloudantNoSQLDB'][0]['credentials']['url'];
     }
    
     // ...
     $app->run();
    
  2. Create the application manifest file. Remember to use a unique host and application name by appending a random string to it (like your initials):

     ---
     applications:
     - name: asthma-tracker-[initials]
     memory: 256M
     instances: 1
     host: asthma-tracker-[initials]
     buildpack: https://github.com/cloudfoundry/php-buildpack
     stack: cflinuxfs2
    
  3. Configure the PHP build pack to use the public/ directory of the application as the Web server directory. Create a $APP_ROOT/.bp-config/options.json file with the following content:

     {
       "WEB_SERVER": "httpd",
       "COMPOSER_VENDOR_DIR": "vendor",
       "WEBDIR": "public",
       "PHP_VERSION": "{PHP_71_LATEST}"
     }
    
  4. Push the application to IBM Cloud:

     cf api https://api.ng.bluemix.net
     cf login
     cf push
    
  5. If you decided to import service credentials from the IBM Cloud environment, use the IBM Cloud dashboard, navigate to the Connections page of the Cloudant service, and create a connection between the service and the application. This action, as shown in the following screen capture, restages the application.

Screen capture of Connections page of the Cloudant service

You should now be able to see the home page for the app at http://asthma-tracker-[initials].mybluemix.net. If you don’t see it, troubleshoot the problem using the tips at vikram-vaswani.in/weblog/2015/03/19/debugging-php-errors-on-ibm-bluemix/.

Summary

This tutorial discussed specific tools, technologies, and APIs. Although these tools and APIs might change and evolve over time, the general principles remain valid and should serve as guidance for other projects you undertake. Here are some tips to keep in mind:

  • Consider your data requirements carefully before choosing your data storage engine. A document-oriented database imposes fewer restrictions on the structure of your data than a traditional SQL RDBMS. This gives you greater flexibility in designing document content, but at the cost of some functionality, such as built-in SQL functions for sorting, grouping and paging through data. Performing these operations typically requires you to create additional custom views or functions, either at the database or application level.
  • A charting library makes it easier for users to understand patterns in the data. Check that the library you use supports the HTML5 <canvas> element for maximum usability on mobile devices and better drawing performance in modern browsers.
  • Binding IBM Cloud services to your application and sourcing service credentials from the shared environment ensures security, while also giving you the flexibility to adjust application infrastructure without redeploying the application.

The goal of this tutorial was to build an application to store personal asthma readings in a database, automatically integrate external weather conditions with those readings, and then create graphs and charts make it easier to analyze the results. The application runs entirely in the cloud with a cloud database, cloud hosting infrastructure and cloud APIs. The Cloudant database service and Weather Company Data API, coupled with the PHP CloudFoundry buildpack, demonstrate how easy it is to build data-driven web applications in IBM Cloud that integrate data from third-party APIs without infrastructure security or scalability concerns.

If you’d like to learn more about the services and tools discussed in this article, start by trying out the live demo of the application. Remember that this is a public demo, so you should be careful not to store confidential information in it. Then, download the code from the GitHub repository, take a closer look to see how it all fits together, and start modifying it to meet your own requirements. Happy coding!

Vikram Vaswani