Overview

Skill Level: Any Skill Level

This tutorial assumes basic familiarity with deploying Cloud Foundry applications to Bluemix

This tutorial illustrates how to use Object Storage to safely backup database documents from CouchDB, like Cloudant. It provides insights for customizing the application and shows how these backups can be used to restore a database in disaster recovery.

Ingredients

Step-by-step

  1. Set up Cloudant and Object Storage Service Instances

    The sample application supports reading credentials from both CouchDB and Object Storage, but works with remote sources as well Рthe credentials for these will need to be added in the manifest. 

    If using Cloudant in Bluemix, first create a Cloudant service instance in the space where the application will be deployed.  Launch the service, and create a new database.

    If using Object Storage for Bluemix, first create an Object Storage service instance in the space where the application will be deployed.

    These new service instances will need to be bound to the application once it is pushed into Bluemix.

     

  2. Download the sample application

    The following steps assume that the code will be modified locally, and pushed to bluemix using the CF CLI.  If using Bluemix DevOps Services, reference the tutorial in order to mirror the steps below.

    The sample app can be found by running the following command the following repository:

    >git clone https://github.com/christiancompton/backup-cloudant.git

    How does the sample application work?

    The primary mechanism in this application is a call to the couchbackup node module, described in depth in this couchbackup tutorial, which gets all documents from the specified database, and stores them in a .txt file. Once complete, the file is uploaded to the specified Object Storage, with the name of the file as the timestamp of the transaction, inside of a container named by the YEAR-MONTH convention.
    The default is to run the backup job once every 24 hours, or every 86400000 milliseconds, but this is easy to modify in the manifest.

     

  3. Make Desired Changes

    The manifest.yml file contains properties that the application needs to run and which can be easily customized. 

     Since the namespace in bluemix is global, make sure to give the application a unique host name.

    Modify the database_name field to point to the name of the database to be backed up. Make sure to supply the cloudant_url for your CouchDB if not using a bound Cloudant, and the Object Storage (all 5 os_ fields) credentials if not using a bound Object Storage for Bluemix service instance. These five credentials are neccessary for Keystone Authentication to use Object Storage.

     

  4. Deploy Application to Bluemix

    Login to Bluemix, targeting your specific API.

    Then issue the following command:

    >cf push <name of application from manifest>

    If you are using bound applications – Cloudant or Object Storage, note that the application will fail to start until the desired service instance(s) have been bound to the application and restaged.

    Once the application begins, it will immediately attempt to backup the database, and will also start the time interval countdown to run the next backup.

     

  5. Check that the application has started correctly

    Once the application has started in Bluemix, it will take approximately five minutes to donload and then upload a CouchDB database record with 40,000 documents.

    To verify that this has worked correctly, navigate to the Object Store being used, where there should be and new container and .txt file.

     

  6. Disaster Recovery - Restoration

    In order to restore a database, see the couchbackup tutorial for detailed information. Since restoration should not be a regular occurrence, it is not automated in this application.

    The first step is to get the desired .txt file that will be used to restore – this can be done using the Object Storage user interface, or by using tools that directly connect to the Swift back end to get the file.

    Once you have the file, make sure that the couchdb  node module is installed globally, and run the following command, making sure to specify the environment variable COUCH_URL before running:

    >cat <filename>.txt |  couchrestore --db <name of existing db to restore>

    One very important stipulation – the database that is being restored must exist before this command will work. If the database does not exist, the restore will not work. Additionally, it is best practice to restore to a new, empty database, rather than attempt to overwrite an existing one.

     

  7. Happy Developing!

    Please feel free to reachout with questions or concerns, and issue pull requests against the sample app with improvements.

    Check out these Object Storage tutorials to learn more about how to use this service in Node.js, PHP, Java, and Python.

    I can be reached via Twitter @cbcompton1.

1 comment on"Object Storage Cloudant Backup"

  1. Christian Compton January 13, 2017

    The sample app was updated on 1/13/2017 to address issues with Cloud Foundry’s Diego architecture.

Join The Discussion