When developing an application that handles geospatial data, it is imperative that we can actually view the data on a map, and see how it is being manipulated by the application.

In this article, I am going to demonstrate how you can easily write a Streams application to display your application geospatial data onto a map in a web browser.

MapViewer Sample

The MapViewerSample from the Github Samples repository is developed  to show how this can be done.  The sample application randomly generates location data for two types of entities around Hong Kong Island.  Taxi data will be displayed with a red marker on a map.  Public Minibus data will be displayed with a green marker.  Here’s a screen capture that shows the end result of our sample application:



The code for this sample application can be found here:   https://github.com/IBMStreams/samples/tree/master/Geospatial/MapViewerSample

Building Blocks

The following diagram shows the major building blocks of the the MapViewerSample application:


The application contains the following components:

  1. A data source that produces geospatial data to be displayed on a map.  In the case of the MapViewer sample, we are simply generating random data points around Hong Kong.  In a real application, this can come from GPS data from mobile phones, smart cars, etc.  The data source, at a minimum, needs to provide the following information:
    • Entity ID – ID of the entity to be displayed on the map
    • Location data – in the form of Well-Known Text
  2. HTTPTupleView operator – The HTTPTupleView from the com.ibm.streamsx.inet toolkit provides the following capabilities in the application:
    1. Ability for us to query for tuple data feeding into the HTTPTupleView operator via Http REST API.
    2. Embedded Jetty server – When the operator is started, a Jetty server is started that allows us to query for tuple data.  In addition, the Jetty server can also serve static HTML files.
  3. map.html – This is the static HTML file to be served by the embedded Jetty server.  map.html contains Javascript that periodically queries for latest tuple data from HttpTupleView using the Http REST API.  Then it uses OpenLayer APIs to parse the tuple data, and display the location data on a map.

Data Source and the Main application

This is the SPL code from the Main composite of the sample application.  This application uses a Custom operator to randomly generate location data.  The custom operator generates the following information:

  • Id of the entity – randomly generated based on number of entities required.
  • Latitude  – randomly generated based on a set min and max values
  • Longitude – randomly generated based on a set of min and max values

With the latitude and longitude information, the application calls the com.ibm.streams.geospatial.ext::point(…) function to convert the coordinates to a WKT string.   The tuples are submitted to a composite operator called MapViewer.

/** Main composite application to generate location data and submit to MapViewer.
* Each tuple has the following attributes:
* * id - id of the entity to be displayed 
* * wkt - geometry of the entity specified as WKT string
* * updateAction - tells the Javascript how to update the entity.  1 to add or update, 0 to remove.
* * note - additional information tagged with the entity.  Note is displayed as a popup on the map.  
* * markerType - allows us to control the color of the entity markers on the map.
composite Main 
     (stream<rstring id, rstring wkt, uint32 updateAction, rstring note, MARKER_TYPE markerType> GreenEntities_out0) as GreenEntities = Custom()
            onProcess :
                 while(! isShutdown())
                     // Randomly generate entities in Hong Kong
                     float64 latitude = randomLatitude(22.248429, 22.282425) ;
                     float64 longitude = randomLongitude(114.137821, 114.236355) ;
                     rstring id = randomId(10, 10) ;

                     // convert coordinates to wkt string
                     rstring wktGeometry = point(longitude, latitude) ;

                     // submit to MapViewr, set upactionAction to 1 to add point to the map
                     submit({ id = id, wkt = wktGeometry, updateAction = 1u, note =
                     "Public Mini Van: " + wktGeometry, markerType = GREEN },
                      GreenEntities_out0) ;
                     block(0.01) ;

         () as MapViewer_2 = MapViewer(RedEntities_out0, GreenEntities_out0)

MapViewer Composite

The most important part of this application is the MapViewer composite.  This composite operator wrappers a HTTPTupleView operator and sets up the resource and context paths for the Jetty server.

public composite MapViewer(input stream<MapViewerT> In0 )
        expression $windowSize : 1 ;
        () as data = HTTPTupleView(In0 as inPort0Alias)
                inPort0Alias : sliding, count($windowSize), count(1), partitioned ;
                contextResourceBase : getThisToolkitDir() + "/etc" ;
                context : "map" ;
                partitionKey : "id" ;
                placement : partitionColocation("mapviewer") ;

The MapViewer composite expects schema of the input stream to be of type MapViewerT.  MapViewerT is defined as follows:

/** The expected schema for the MapViewer composite
 * id - the id of the entity
 * wkt - geometry of the entity specified in WKT format
 * updateAction - 1 to add / update the entity in the map, 0 to remove the entity from the map.
 * note - information to be displayed in the popup 
 * markerType - if the entity is a point, the color of the marker to use.  MARKER_TYPE can be RED, GREEN OR YELLOW.
type MapViewerT = rstring id, rstring wkt, uint32 updateAction, rstring note, MARKER_TYPE markerType ;

In the composite, the HTTPTupleView operator is set up with a sliding window, partitioned by the entity ID.  This allows us to keep history for each of the entities in case we want to show where the entities have traveled over a period of time.  The window size is configurable via the $windowSize parameter.

The HTTPTupleView operator is also set up with contextResourceBase and context parameter.  The contextResourceBase parameter defines the directory location of resources that will be available through the URL context defined by the context parameter.  In this case, it is expected that the static html resources are located in <Applicaton Directory>/etc directory.

The context parameter defines a URL context path that maps the resources defined by the contextResourceBase.

With this configuration, you can browse for static html files from the Streams application using the following URL:  http://<PE host>:8080/map


Next, we need an HTML page for displaying the data.  The map.html file is located in the MapViewerSample/etc/ directory.  It contains JavaScript that displays a map, queries for data from the Streams application and displays the data as markers on the map.

To do this, here are some of the important parts of this html file:

  • Import OpenLayer.js from www.openlayers.org – To have access to the OpenLayers Javascript APIs, import the following script:
 <div id="test"></div>
 <div id="map-canvas"></div>
 <script src="http://www.openlayers.org/api/OpenLayers.js"></script>
  • Set up OpenLayer map and layers in the initialize function.  In this function, we set up three layers.  At the base layer is a map canvas.  The markers layers is for displaying any POINT data on the map.  The polygon layer is for displaying any POLYGON data on the map.  We then call “loadData” to get the data from the Streams application.
function initialize() {
    var map_options = {
    div : this.mapDiv,
    allOverlays : false,
    maxExtent : this.mapExtent,
    controls : [ new OpenLayers.Control.DragPan(),
        new OpenLayers.Control.Navigation(),
        new OpenLayers.Control.PanZoomBar(),
        new OpenLayers.Control.ScaleLine(),
        new OpenLayers.Control.MousePosition(),
        new OpenLayers.Control.LayerSwitcher() ]
    map = new OpenLayers.Map('map-canvas', map_options);
    map.addLayer(new OpenLayers.Layer.OSM());

    // create marker layer
    markers = new OpenLayers.Layer.Markers("Markers");
    // create polygon layer
    vectors = new OpenLayers.Layer.Vector("Polygon", {
         styleMap: new OpenLayers.StyleMap({
        "default": new OpenLayers.Style({
        fillColor: "#33CC00",
        strokeColor: "#000000",
        strokeWidth: 1
    // retrieve data from HTTPTupleView
  • In the loadData method, the method constructs the URL for querying the latest data from input port 0 of the HTTPTupleView operator. The URL to use for fetching data is:  http://[PE Host]:8080/map/data/ports/0/tuples
  • Next, we construct an HTTP request with the constructed URL.  When a response is received, we call the updateMap method with the response text from the request.
  • At this time, we also schedule a timeout for the page.  loadData will be called again after the timeout period to refresh the markers on the map.
function loadData() { 
    // retreive data from input port of HTTPTupleView 
    var url = window.location.protocol + "//" + window.location.host + "/map/data/ports/input/0/tuples";

    // construct HTTP request and send
    var markerReq = new XMLHttpRequest();
    markerReq.open("GET", url, true);

    markerReq.onreadystatechange = function() {
        // when we get the response back, update marker and polygon locations
        if (markerReq.readyState == 4 && markerReq.status == 200) {
    markerReq.onLoad = markerReq.send(null);

    // refresh data every x second based on period parameter
    timeoutID = setTimeout('loadData()', getPeriod());
  • In the updateMap(response) method, the response text will be encoded as JSON.  We will first parse the response and construct objects from the JSON text.
  • Location data of the entity will be encoded as WKT.  We will use the OpenLayers WKT formatter to parse the WKT string from tuple data.  The formatter returns a Feature.  We need to tranform the feature coordinate from WGS 1984 to Sperical Mercator Projection before it can be displayed on the map.
 function updateMap(response) {
    try {
        // HTTPTupleView retuns tuples information as JSON, parse the JSON into a list of objects to process
        var allObjects = JSON.parse(response);
         // for each object, update marker or poly accordingly. 
         for (var i=0; i<allObjects.length; i++) {
         var markerID = allObjects[i].id;
         var wkt = allObjects[i].wkt;
         var markerType = allObjects[i].markerType;
         var updateAction = allObjects[i].updateAction;
         // construct WKT formatter to parse out geometry from tuples
         var formatter = new OpenLayers.Format.WKT();
         // parse wkt string from tuples
         var feature = formatter.read(wkt);
         // transform from WGS 1984 to Spherical Mercator Projection
         var transformedFeature = feature.geometry.transform(new OpenLayers.Projection("EPSG:4326"), map.getProjectionObject());
         var vertices = transformedFeature.getVertices();
         // this is a point, number of vertices must be 1
         if (vertices.length == 1) {
  • Next we will use the transformed coordinates and create a marker, and add the marker to the Marker Layer of the map.
    // create longlat object for marker
    var myLongLat = new OpenLayers.LonLat(transformedFeature.x, transformedFeature.y);
    // if updateAction > 1, add marker to map
    if (updateAction > 0)
        var icon = new OpenLayers.Icon(getIcon(markerType));
        var marker = new OpenLayers.Marker(myLongLat, icon.clone());

 Putting This All Together

To view the location data on a map in a browser, you need to first launch your application to a Streams instance.  Once the application is successfully started, start a web browser and enter the following URL:

http://[PE Host]:8080/map/map.html

What’s happening under the cover:

  • When the Main SPL application is started, the HTTPTupleView operator starts an embedded Jetty server, waiting for HTTP requests.
  • The Custom operator from the Main composite generates random location data and submits it to HTTPTupleView operator.
  • The HTTPTupleView operator  put the tuples into a sliding window.  The window is partitioned by entity ID.
  • When map.html is requested from a web browser, the Jetty server serves the static map.html page from the Streams application.
  • When the web page is initialized, the java script loads the map onto the browser.  Next it constructs an HTTP request with the following URL to get the latest data from the HTTPTupleView operator:  http://[PE Host]:8080/map/data/ports/0/tuples.
  • When the Jetty server receives the HTTP request, it processes the request, fetches the correct data and returns the latest data in the form of JSON.
  • When map.html receives a response back, it decodes the JSON response.   From the response, fetches and parses the WKT string into the correct coordinate system.
  • For each tuple / entity received from the response, we add a marker to the Marker Layer on the map.

What about popup?

In the previous section, we mentioned that each of the entities can be tagged with a note.  The note information can be displayed in popup on the map.  To see this information, use the following URL:

http://[PE Host]:8080/map/map.html?popup=true

You should see the following in the web browser with the popup parameter enabled:


Join The Discussion