Contents


Overview

Skill Level: Intermediate

Some basic knowledge of Bluemix, AWS and Node-RED

This recipe describes how a user can make use of Node-RED on an Amazon EC2 (Elastic Compute Cloud) instance to make calls to services provided by multiple cloud platforms. For example, Amazon S3 Buckets and IBM Bluemix Weather Company data.

Ingredients

1.  IBM Bluemix account

2.  Amazon Web Services account

Step-by-step

  1. Deploy Node-RED on an Amazon EC2 Instance

    Before you can deploy Node-RED, you must first create an EC2 instance using your AWS account. ¬†Follow the instructions found here on nodered.org to create the EC2 image and setup Node-RED. ¬†If you are not sure how to connect to your EC2 instance after it is created, check out the AWS guide for connecting to your instance. ¬†Once connected to your EC2 instance, follow the steps to “Setup Node-RED” but do not start Node-RED yet. ¬†Before starting Node-Red, run the following commands to install the “node-red-node-aws” and “node-red-contrib-web-worldmap” nodes:

    sudo npm install -g node-red-node-aws

    sudo npm install -g node-red-contrib-web-worldmap  

    Now that we have some handy nodes installed, it is time to start Node-RED.  Use the following command to start Node-RED:

    node-red

    If everything was successful, you should now be able to open Node-RED in your browser by going to: 

    http://<your-instance-ip>:1880/

    Check to make sure the amazon s3 and worldmap nodes are present:

    8-3-2016-1-47-44-PM

    You will come back to Node-RED later in this recipe. ¬†For now, let”s move onto the next step and create an Amazon S3 Bucket.

  2. Create an Amazon S3 Bucket, Upload a File with GPS Coordinates and Create and Access Key

    Amazon S3 is a place to store files on AWS. ¬†You will create a “Bucket” to hold a file with some GPS coordinates which can then be used to gather weather forecast information using the Weather Company service on IBM Bluemix. ¬†To create a bucket, go to¬†https://console.aws.amazon.com/s3¬†and click the Create Bucket button. ¬†Give the bucket a name and a region and click the Create button. ¬†Make note of the bucket name and region as you will need those later in this recipe. ¬†Once the bucket is created, click on the bucket name to open the bucket.

    8-3-2016-2-25-48-PM

    Now click the Upload button and add this file which contains some GPS coordinates.

    8-3-2016-2-46-03-PM

    You now have a bucket that contains a file with GPS locations.  Next you need to create an access key so that you can programmatically access the bucket from Node-RED.  In AWS, click on your user name in the upper right hand corner and choose Security Credentials.

    8-3-2016-6-01-05-PM

     

    Expand the section for Access Keys and click the Create New Access Key button.  Download the access key file and save it for use later in this recipe.

    8-3-2016-6-02-16-PM

     

    Let”s now move onto the next step and create an instance of the Weather Company Data on IBM Bluemix.

  3. Create an Instance of the Weather Company Data Service on IBM Bluemix

    In this step, you will create an instance of the Weather Company Data on IBM Bluemix. ¬†Among many other features, this service provides APIs to gather weather forecasts for locations that you specify. ¬†First, navigate to the Bluemix Catalog and login if you are not already. ¬†You can quickly find the Weather Data service by typing in “weather” in the search field.

    8-3-2016-1-08-07-PM

    Click on the Weather Company Data service and either use the default names for the service and credentials or provide your own.  Then scroll to the bottom and click the Create button.

    8-3-2016-1-09-09-PM

    8-3-2016-1-09-36-PM

    Once the service is created, click on the Service Credentials tab and make note of your username and password.  Mark these down somewhere as you will need them later in this recipe.

    8-3-2016-1-11-18-PM

    You now have an instance of the Weather Company Data service created. ¬†Next, we”ll move back to Node-RED and see how we can piece all this together in a Node-RED flow.

  4. Create a Node-RED Flow

    In this step, you will import a Node-RED flow and configure some nodes with information from your S3 bucket and Weather Data service.  The flow will first access your locations file on AWS S3, parse the GPS locations within that file, call the Weather Data service for each location and then finally plot those locations and their forecasts on a world map.

    Navigate back to Node-RED where you ended in step one http://<your-instance-ip>:1880/

    Use the Node-RED menu in the upper right hand corner to import from the clipboard:

    8-3-2016-3-33-15-PM

    Copy and paste the text from this file into the clipboard box.  The text should not have any new lines in it so be sure to download the file, open it in a text editor, make sure word wrapping is turned off and select all the text and copy it.  Finally, click the Import button.

    8-3-2016-5-45-47-PM

    Place the imported flow into the flow editor:

    8-4-2016-11-08-19-AM

    You now need to configure a couple of the nodes with your S3 and Weather Data information. ¬†Double click on the “s3 location-bucket” node. ¬†Click on the pencil to the right of the AWS field. ¬†Enter your access keys found in the file you downloaded in step 2 of this recipe.

    8-3-2016-6-14-39-PM  8-3-2016-6-13-34-PM

     

    Enter the Bucket, Filename and Region to match your information from step two of this recipe and click done.

    8-3-2016-6-19-52-PM

    Next, you will configure the “Get Forecast” node with your credentials for the Weather Data service from step three. ¬†Double click the “Get Forecast” node. ¬†Check the Use Basic Authentication box. ¬†Place in your user name and password for the Weather Company Data service you acquired in step three. ¬†Finally click Done.

    8-3-2016-6-25-33-PM

    You are now configured and ready to run your flow.

     

  5. Deploy and Execute your Node-RED Flow

    You will now deploy and execute your Node-RED flow.  Click the Deploy button in the upper right hand corner.

    8-3-2016-6-33-35-PM

    We should now be able to navigate to the world map.  In a separate web browser tab, go to the following link:

    http://<your-instance-ip>:1880/worldmap

    The world map will not show any pins as of yet since we have not executed the flow. ¬†The flow is setup to automatically run every five minutes. ¬†To execute the flow on demand, click the little blue tab to the left of the “Timestamp” node.

    ***Be aware that the free version of the Weather Company Data service only allows for 10 API calls per minute.  With that Node-RED flow you are making six API calls every time you execute the flow.  An API call is made for each GPS coordinate.***

    8-4-2016-10-40-24-AM

    Now examine the world map again.  You should now see a number of pins plotted.  Clicking a pin will show the forecast.

    8-4-2016-10-39-21-AM

    ¬†If it is not working for any reason, try using the Node-RED “Debug” nodes and tab to see if there are any error messages.

    You can try updating your locations.txt file in your S3 box with other GPS locations.  Just make sure there are no extra new lines at the end of the file.

  6. Understanding the Solution Architecture and Node-RED Flow

    This step provides an explanation of the solution architecture and each node in the Node-RED flow.

    Explanation of solution architecture:

    In the picture below, you can see that Node-RED is running on an EC2 instance on the Amazon Web Services cloud platform.  The Node-RED flow grabs GPS coordinates from a file in an S3 bucket in the AWS cloud.  Node-RED flow then makes REST API calls using those GPS coordinates to an instance of the Weather Company Data service in the IBM Bluemix cloud.  Lastly, Node-RED uses the World Map node to plot the location on a map with weather forecast information.

    8-4-2016-11-41-11-AM

    Explanation of Node-RED flows:

    timestamp – Executes the flow every 5 minutes or on demand when the tab is clicked

    s3 bucket – Grabs a file from an S3 bucket and returns the contents as a buffer

    Buffer to String – Converts the contents of the buffer to a string

    Split on new line РSplits the string into multiple messages based on the new line char.  Executes the remaining flow for each substring.

    Build URLs – Forms the needed URL with the correct geocode in order to call the Weather Company Data REST API

    Get Forecast – HTTP GET request using the URL passed in in order to get weather forecast form Bluemix service

    Format JSON – converts JSON string to JS object

    Add WorldMap Data – Function that composes the message payload to have the name, lat, lon and forecast info.

    Complete Forecast Message – Some more parsing to format the message before passing into World Map node.

    Send to WorldMap – Create the world map page and plots the point on the map with forecast info.

2 comments on"Deploy Node-RED on Amazon EC2 to Read GPS Coordinates from an S3 Bucket and Plot Weather Company Data on a World Map"

  1. vikonline June 26, 2017

    Hi I-Logix_Disciple – Thanks for the above literature. I have a below requirement that I am trying to solve using Node-RED

    I have lat/long coordinated of transformers+transmission towers (100+)+sub station transformers etc., across the US
    I could get them plotted on the world map as shown above in Bluemix(r)
    However, can I connect all the assets as a linear line to show a grid? + Can I identify sub-regions in this grid with a different color?

    • I-Logix_Disciple June 29, 2017

      According to the following link: https://flows.nodered.org/node/node-red-contrib-web-worldmap

      To connect the assets with lines:

      Areas and Lines
      If the payload contains an area property Рthat is an array of co-ordinates, e.g.
      [ [51.05, -0.08], [51.5, -1], [51.2, -0.047] ]

      then rather than draw a point and icon it draws the polygon. Likewise if it contains a line property it will draw the polyline.
      iconColor : can set the colour of the polygon or line.
      name : is used as the id key Рso can be redrawn/moved.
      layer : declares which layer you put it on..

      To identify subregions with different colors:

      You could use circles, heat maps or icons with specific colors. All 3 options are mentioned in the link. Hope this helps.

Join The Discussion