IBM Developer Advocacy

Location Tracker – Part 1



markwatson
6/14/16

Offline apps with mapping, Swift and cloud sync

Check out part 2 of our Location Tracker series! —The editors.

The goal of the Location Tracker app is to show Swift developers how simple it is to use Cloudant to track, store and query locations, all while enabling offline first design and providing architecture guidance for scaling your solutions to support millions of users.

In this tutorial we’ll show you how we built the app and what strategies we employed to accomplish our goal. We’ll show you how we used Cloudant Sync for offline support and data synchronization. We’ll show you how we used Cloudant Geo to perform and visualize geospatial queries. Finally, we’ll introduce alternative architectures and approaches that we’ll use in future tutorials to show you how to scale your apps to support millions of users.

Overview

The Location Tracker app is an iOS app developed in Swift that tracks user locations and stores those locations in Cloudant. As a user moves, and new locations are recorded, the app queries the server for points of interests near the user’s location.

Below is a screenshot of the Location Tracker app. Blue pins mark each location recorded by the app. A blue line is drawn over the path the user has travelled. Each time the Location Tracker app records a new location a radius-based geo query is performed in Cloudant to find nearby points of interest (referred to in the app as “places”). The radius is represented by a green circle. Places are displayed as green pins:

location tracker screenshot

Requirements

To help achieve our goal we created 5 key requirements:

  1. Track location in the foreground and background: The app should be able to track a user’s location when the app is running in the foreground or the background.
  2. Use geospatial queries to find points of interest within a specified radius: The app should show users how to use Cloudant Geo to perform geospatial queries.
  3. Run offline: The app should be able to track user locations while offline and sync those locations to Cloudant when a network connection is available.
  4. Keep user location information private: Users should not have access to other users location information.
  5. Provide ability to consolidate and analyze all locations: It should be simple for backend engineers or data scientists to perform analysis on all locations without compromising requirement #4.

Architecture

In order to satisfy requirement #4 (User Privacy) the Location Tracker was implemented using the database-per-user design pattern. A dedicated database is created for each user that only that user has access to. Here’s how it works:

  1. When the user registers, the Location Tracker app posts the user information to our Node.js server.
  2. The Node.js server creates a new user in the Users database.
  3. The Node.js server creates a user-specific database to track the user’s locations.
  4. The Node.js server returns the database name and authentication information to the app.
  5. The app connects directly to Cloudant to sync location information.

We’ll discuss this in more detail later. Here is a high-level diagram of the system architecture:

location tracker architecture

Location Tracker uses the Cloudant/CouchDB “one-database-per-user” design pattern.

In addition to creating user-specific databases as shown in the adjacent architecture diagram, the server also configures continuous replication for each user-specific database into a consolidated database (All Locations). This satisfies requirement #5 (Location Consolidation & Analysis) by providing us a single location to query and analyze all locations recorded by all users while not compromising user safety and privacy (no users are given direct access to the consolidated database).

Note: The database-per-user design pattern makes it easy to sync location information between the iOS app and Cloudant, all while ensuring that information is kept private. It is a great solution for small- to medium-sized apps. In the next tutorial we will show you alternative ways of replicating user-segregated data and how you can scale your app to support millions of users.

The server

The Location Tracker Server is a Node.js application that provides RESTful APIs for registering new users and querying places using Cloudant Geo. When you install the Location Tracker Server three databases will be created in your Cloudant instance:

cloudant dashboard for location tracker dbs
  1. lt_locations_all – This database is used to keep track of all locations. When a user registers, a specific database will be created to track locations for that user. Each user-specific database will be configured to continuously replicate into the lt_locations_all database.
  2. lt_places – This database contains a list of places that the Location Tracker app will query.
  3. lt_users – This database is used to manage users. Each user will have a username, password and information regarding the location database for that specific user.

The lt_locations_all and lt_places database will each be created with a geo index allowing you to make geo queries and take advantage of the integrated map visuals in the Cloudant Dashboard. The lt_places database will be populated with 50 sample places that follow the path of the “Freeway Drive” debug location setting in the iOS simulator:

Location Tracker's sample data can be previewed directly in the Cloudant dashboard with integrated Mapbox tiles.

Location Tracker’s sample data can be previewed directly in the Cloudant dashboard with integrated Mapbox tiles.

Follow the instructions on the Location Tracker Server GitHub page to get the Location Tracker Server up and running locally or on Bluemix.

The client

The Location Tracker client is an iOS app developed in Swift. As mentioned previously the Location Tracker app tracks and records user locations and queries Cloudant for points of interest. When a new user registers with the Location Tracker app a new database will be created specifically to track locations for that user.

locationtracker_locations-user-db

The Location Tracker app uses Cloudant Sync for iOS to store locations locally and sync them to Cloudant:

Geolocation data previewed in the Cloudant dashboard, and the same data rendered in the Location Tracker UI.

Geolocation data previewed in the Cloudant dashboard, and the same data rendered in the Location Tracker UI.

Follow the instructions on the Location Tracker App GitHub page to get the Location Tracker App up and running in Xcode.

How it works

Hopefully at this point you have successfully deployed the Location Tracker Server and can run the Location Tracker app in the iOS simulator or on your iOS device.

In the rest of this tutorial we’ll go into more detail on how the app works and how we satisfied our 5 key requirements, including:

  • How we track a user’s location in iOS.
  • How we use the database-per-user design pattern to segregate and sync user locations.
  • How we use Cloudant Sync to support offline location tracking and two-way synchronization with Cloudant.
  • How we replicate user locations into a consolidated location database.
  • How we use Cloudant Geo to find points of interest near a user’s location.

User registration

It all starts with user registration. As you can see below we only require a username and a password. You can easily extend the app to add new fields, such as name, email address, etc.

locationtracker_user-reg

When the user taps the Register button, the app executes an HTTP PUT to the Location Tracker Server. The PUT body is a JSON representation of the user:

{
    "username": "markwatson",
    "password": "passw0rd",
    "type": "user",
    "_id": "markwatson"
}

The JSON is generated in the getRegisterHttpBody function in the RegisterViewController. As you can see below we are simply creating a dictionary and using the built-in NSJSONSerialization class:

 func getRegisterHttpBody(_id:String) -> NSData {
    var params: [String:String] = [String:String]()
    params["username"] = self.usernameTextField.text
    params["password"] = self.passwordTextField.text
    params["type"] = "user"
    params["_id"] = _id
    var body: NSData!
    do {
        body = try NSJSONSerialization.dataWithJSONObject(params as NSDictionary, options: [])
    }
    catch {
        print(error)
    }
    return body
}

The real work begins when the Node.js server receives the PUT request. The request is processed in the createUser function in api/routes.js and the following steps are executed:

  1. Check if the user exists with the specified id. If the user already exists then return a status of 409 to the client.
  2. Create a location database for the user. The database will be called lt_locations_user_USERNAME and will be used to store only locations for this user. On login, the name of the database will be returned to the application to allow for synchronization using Cloudant Sync.
  3. Create geo indexes on the newly created database.
  4. Generate an API key and password in Cloudant used to access the database. The API key and password will also be returned to the application on login.
  5. Associate the API key with the newly created location database.
  6. Store the user in the users database with their id, password, api key, and api password (the api password will be encrypted).
  7. Configure continuous replication for the user’s location database to the lt_locations_all database.

Here’s the javascript code. Refer to the routes.js file for the full definition of each function:

 var username = req.params.id;
var dbName = 'lt_locations_user_' + encodeURIComponent(username);
checkIfUserExists(cloudant, req.params.id)
  .then(function () {
    return createDatabase(cloudant, dbName);
  })
  .then(function () {
    return createIndexes(cloudant, dbName);
  })
  .then(function () {
    return generateApiKey(cloudant);
  })
  .then(function (api) {
    return applyApiKey(cloudant, dbName, api);
  })
  .then(function (api) {
    return saveUser(req, cloudant, dbName, api);
  })
  .then(function (user) {
    return setupReplication(cloudant, dbName, user);
  })
  .then(function (user) {
    res.status(201).json({
      ok: true,
      id: user._id,
      rev: user.rev
    });
  }, function (err) {
    console.error("Error registering user.", err.toString());
    if (err.statusCode && err.statusMessage) {
      res.status(err.statusCode).json({error: err.statusMessage});
    }
    else {
      res.status(500).json({error: 'Internal Server Error'});
    }
  });

User login

Users are logged in immediately after registering. The app sends the following request to the Node.js server:

{
    "username": "markwatson",
    "password": "passw0rd"
}

Here is a sample response:

{
    "ok": true,
    "api_key": "ytorestenauneexxxxedstoo",
    "api_password": "ffdc36ea8dbaadxxxx94d9d884d0255c56c08e1e",
    "location_db_name": "lt_locations_user_markwatson",
    "location_db_host": "9f61849d-2884-4463-XXXX-56344789b05c-bluemix.cloudant.com"
}

The response contains the information needed to sync locations with the Cloudant database that was created for this user. These values, along with the user’s login information, are subsequently stored on the device (passwords are stored securely in the Keychain):

UsernamePasswordStore.saveUsernamePassword(username, password: password)

LocationDbInfoStore.saveApiKeyPasswordDbNameHost(
    dict["api_key"] as! String,
    apiPassword: dict["api_password"] as! String,
    dbName: dict["location_db_name"] as! String,
    dbHost: dict["location_db_host"] as! String
)

Storing this information locally allows the app to function completely offline. If a user kills the app while logged in, and re-opens the app while offline, the app will automatically log the user in.

In addition, these values are made available to the application via the AppState class. Any class in the project can access these values at any time. For example:

let credentials = "(AppState.locationDbApiKey!):(AppState.locationDbApiPassword!)"
let url = "https://(credentials)@(AppState.locationDbHost!)/(AppState.locationDbName!)"

We’ll take a closer look at this code later.

Tracking locations

Tracking locations in iOS is fairly straight forward, but tracking locations in the background can be a little tricky. Apple only allows continuous location tracking in the background to be performed by certain types of apps (Fitness, GPS, etc), but they offer significant location changes to all apps (as long as the user approves it). In the Location Tracker we use the significant-change location service when the app is in the background. Refer to the iOS Documentation for more information on location tracking and the significant-change service.

We created a wrapper that automatically handles switching between monitoring real-time locations and the significant-change service. The wrapper is called LocationMonitor. Here is an example of how to use the LocationMonitor:

class MyViewController: UIViewController, LocationMonitorDelegate

override func viewDidAppear(animated: Bool) {
   ...
   LocationMonitor.instance.addDelegate(self)
}

func locationUpdated(location:CLLocation, inBackground: Bool) {
   // do something with the location
}

There are a number of variables that dictate when the LocationMonitor will notify subscribers of new locations. Those variables can be found in the AppConstants class:

static let minMetersLocationAccuracy : Double = 25
static let minMetersLocationAccuracyBackground : Double = 100
static let minMetersBetweenLocations : Double = 15
static let minMetersBetweenLocationsBackground : Double = 100
static let minSecondsBetweenLocations : Double = 15
  1. minMetersLocationAccuracy – The iOS Location libraries report the accuracy of a given location. This variable dictates how accurate a location must be while running in the foreground to notify registered subscribers.
  2. minMetersLocationAccuracyBackground – This variable is similar to minMetersLocationAccuracy, but is used when tracking locations in the background.
  3. minMetersBetweenLocations – The iOS Location libraries can report the slightest changes in location. For our purposes we don’t want to store every single location if the user hasn’t moved. This variable dictates the minimum # of meters that the user must have moved to report the location.
  4. minMetersBetweenLocationsBackground – This variable is similar to minMetersBetweenLocations, but is used when tracking locations in the background.
  5. minSecondsBetweenLocations – Similar to minMetersBetweenLocations this variable dictates the minimum # of seconds that must have passed since the last location to report the new location.

Syncing locations

Before we can sync locations to our Cloudant database we need to configure a local datastore. In the viewDidLoad function of the MapViewController class you will see a call to initDatastoreManager. This function initializes the datastore manager which can be used to manage one or more datastores and specifies where the local datastores should reside on the device:

func initDatastoreManager() {
    let fileManager = NSFileManager.defaultManager()
    let documentsDir = fileManager.URLsForDirectory(.DocumentDirectory, inDomains: .UserDomainMask).last!
    let storeURL = documentsDir.URLByAppendingPathComponent("locationtracker")
    let path = storeURL.path
    do {
        datastoreManager = try CDTDatastoreManager(directory: path)

    } catch {
        fatalError("Failed to initialize datastore: (error)")
    }
}

After we initialize the datastore manager we call the initLocationsDatastore function to initialize the datastore for user locations. Here we create a new datastore and index on the created_at property:

func initLocationsDatastore() {
    do {
        locationDatastore = try datastoreManager!.datastoreNamed(locationDatastoreName)
        locationDatastore?.ensureIndexed(["created_at"], withName: "timestamps")
    }
    catch {
        fatalError("Failed to initialize location datastore: (error)")
    }
}

Now we are ready to start saving locations. When a new location is captured we create a new instance of the LocationDoc class:

let locationDoc = LocationDoc(docId: nil,
                              latitude: location.coordinate.latitude,
                              longitude:location.coordinate.longitude,
                              username:AppState.username!,
                              sessionId: AppState.sessionId,
                              timestamp: NSDate(),
                              background: inBackground)

Then we create a CDTDocumentRevision to be stored in the local datastore:

func createLocationDoc(locationDoc: LocationDoc) -> Bool {
    ...
    let rev = CDTDocumentRevision(docId: locationDoc.docId)
    rev.body = NSMutableDictionary(dictionary:locationDoc.toDictionary())
    do {
        try locationDatastore!.createDocumentFromRevision(rev)
    }
    catch {
        print("Error creating location: (error)")
    }
    return true
}

Finally, we sync the local datastore with Cloudant. We start by configuring the URL to the Cloudant database. We use the server and auth information returned by the server and stored in the AppState class:

let credentials = "(AppState.locationDbApiKey!):(AppState.locationDbApiPassword!)"
let url = "https://(credentials)@(AppState.locationDbHost!)/(AppState.locationDbName!)"

Next we create a one-way replication job:

let factory = CDTReplicatorFactory(datastoreManager: self.datastoreManager)

let job = CDTPushReplication(source: self.locationDatastore!, target: url)

// Create the replication job.
var job = try factory.oneWay(job)

// Assign self as the replication delegate (to be notified when the job is complete).
job!.delegate = self

// Start the job
try job!.start()

The code above shows how we can push new locations to Cloudant, but we can also pull locations from Cloudant, and when a user first logs in we do just that. In the viewDidAppear function in the MapViewController we call syncLocations(.Pull). This will create a one-way replication job from Cloudant to the app. When the replication is finished our local datastore will contain the locations retrieved from Cloudant. We call the loadLocationDocsFromDatastore function to load the locations and add them to the map. Here is the loadLocationDocsFromDatastore function:

func loadLocationDocsFromDatastore() {
    let query = ["created_at": ["$gt":0]]
    let result = locationDatastore?.find(query, skip: 0, limit: UInt(AppConstants.locationDisplayCount), fields:nil, sort: [["created_at":"desc"]])
    guard result != nil else {
        print("Failed to query for locations")
        return
    }
    dispatch_async(dispatch_get_main_queue(), {
        self.removeAllLocations()
        // we are loading the documents from most recent to least recent
        // we want our array to be in the oppsite order
        // so we can draw our path and when we add new locations we increment the label
        // here we enumerate the documents and add them to a local array in reverse order
        // then we loop through that local array and add them one by one to the map
        var docs: [CDTDocumentRevision] = []
        result!.enumerateObjectsUsingBlock({ (doc, idx, stop) -> Void in
            docs.insert(doc, atIndex: 0)
        })
        for doc in docs {
            if let locationDoc = LocationDoc(aDoc: doc) {
                self.addLocation(locationDoc, drawPath: false, drawRadius: false)
            }
        }
        self.drawLocationPath()
    })
}

This same function is called when a user logs in while offline. This allows the user to see any locations store in the local datastore event when the network is unavailable.

Querying places

As new locations are recorded the Location Tracker app looks for points of interest (“places”) within radius of those locations. Places are stored in the lt_places database with a GeoJSON geometry object. Here is a sample place:

{
  "_id": "94cf2fcb31459b2244bf8ff6140a5282",
  "_rev": "1-6ded3878845982c77e08c19bfed65d95",
  "geometry": {
    "type": "Point",
    "coordinates": [
      -122.3162468,
      37.4722645
    ]
  },
  "name": "Edgewood Park Natural Preserve",
  "type": "Feature",
  "created_at": 1462910143508
}

The lt_places database also includes a Cloudant geospatial index which allows us to perform geo-based queries. The index is created when you install the Location Tracker server and is defined as follows:

{
  "_id": "_design/points",
  "_rev": "1-cd7d85016e88bcc571b7d6e8c2d33768",
  "language": "javascript",
  "st_indexes": {
    "pointidx": {
      "index": "function (doc) { if (doc.geometry && doc.geometry.coordinates) { st_index(doc.geometry); }}"
    }
  }
}

Cloudant Geo supports a wide-range of geo-based queries (click here for more information). We use a simple radius-based query in the Location Tracker app. The app sends the query to the Node.js server which in turn sends the query to Cloudant. Here is how the app formulates the query:

let url = NSURL(string: "(AppConstants.baseUrl)/api/places
    ?lat=(lastLocation.geometry!.latitude)
    &lon=(lastLocation.geometry!.longitude)
    &radius=(AppConstants.placeRadiusMeters)
    &relation=contains
    &nearest=true
    &include_docs=true")

You can see above we are connecting to the /api/places endpoint on the Node.js server. We are passing the latitude and longitude of the last location and a radius defined in AppConstants. We are specifying a geometric relationship of contains which tells Cloudant to return any object that is within the radius of the specified lat/long. Finally, we are requesting that Cloudant return documents in order of closest to furthest (nearest=true).

The Node.js server calls the geo-index endpoint on Cloudant (passing in the query string sent from the Location Tracker app). See the getPlaces function in api/routes.js:

...
var url = cloudant.config.url + "/lt_places/_design/points/_geo/pointidx";
url += "?" + querystring.stringify(req.query);
request.get({uri:url}, function(err, response, body) {
...

When the app receives the list of places from the server it stores them in a local datastore, just like we did with locations. This allows us to view the places we have queried while running offline.

Conclusion and next steps

In this tutorial we showed you how to create user-specific Cloudant databases on the fly to segregate locations for individual users. We showed you how to track locations in iOS and how to use Cloudant Sync to sync locations with Cloudant and track locations while running offline. We touched on how you can programmatically configure continuous replication to replicate locations in user-specific databases to a central database for analyzing all locations. Finally, we showed you how to use Cloudant Geo to index and query documents within the radius of a lat/long and view locations on maps inside the Cloudant Dashboard.

In doing so we satisfied our 5 key requirements:

  1. Track location in the foreground and background.
  2. Use geospatial queries to find points of interest within a specified radius.
  3. Run offline.
  4. Keep user location information private.
  5. Provide ability to consolidate and analyze all locations.

In future tutorials we will discuss alternative ways to satisfy these requirements and how you can use Cloudant to support millions of users, including:

  • How to use the CouchDB change feed as an alternative to continuous replication.
  • How to use Cloudant Envoy to store locations in single database while maintaining user privacy and safety.
  • How to use alternative map providers, such as Mapbox, and how to store maps offline.

blog comments powered by Disqus